The Infinite Power of Finite Language

Published: April 2025

This article is just my way of appreciating some of the fascinating work Chomsky did in linguistics - particularly in the area that’s become wildly relevant in 2025: computational linguistics.

Chomsky believed there is something inherently special about humans when it comes to our relationship with language. Why is it that no animal – no matter how intelligent or how well it can perform in various tests (chimpanzees beat humans in memory tests! Though to be fair, I’d probably lose to a goldfish) – can grasp the concept of human language? And why is it that human babies, arguably some of the most useless, high-maintenance creatures ever designed (relying entirely on extreme cuteness in order to survive) seem to learn it at an incredibly quick pace, with little to no formal instruction?

The truth is, it’s not surprising that there’s something setting humans apart. After all, in a world filled with a million and one things that can easily kill us, we somehow managed to thrive. And although we didn’t get there just by talking our way out of a lion’s den, communication was absolutely crucial in the process, and it’s often attributed as the most probable reason we were able to take over the globe. The scale at which humans can work together to build things, govern societies, make decisions, etc. is simply unmatched by any other creature. And the key factor in all of this is language.

Hopefully this is enough to convince you just how important language is. With the rise of Artificial Intelligence this decade, it's worth noting how much of it still comes back to language – and how even the most advanced systems still struggle to grasp the thing we do effortlessly: communicate.

ChatGPT and the many other LLMs clearly had a massive jump in terms of capability, but they’re far from new. In the late 1960s, SHRDLU was a success – a toy model that understood natural language and allowed users to interact with blocks in a simulated world. Unfortunately, its success was short-lived due to the fact that it wasn’t scalable beyond its extremely limited setup. Even though it served to demonstrate what AI might one day be capable of, the technology to properly and comprehensively process natural language was still far away.

SHRDLU was an example of a symbolic AI – and it relied on strictly defined rules, symbols, and logical structures, attempting to convert natural language into machine-friendly logic, as opposed to today’s extremely complex pattern recognition via neural networks. Though symbolic AI was not based on Chomsky’s models directly, the notion of trying to fit the infinitely complex phenomenon of ‘natural language’ into a formal system of rules was deeply influenced by his work, most notably in his book Syntactic Structures (1957). Although this approach eventually proved too limited to be useful in real-world AI applications, it laid down the foundation for computational linguistics.

Chomsky also developed the idea of Universal Grammar. He believed every human is born with a grammatical blueprint that allows us to acquire language, and this language faculty is innate – hence why learning languages is often an unconscious process. Babies don’t attend language classes, and reading about a grammatical rule is rarely enough to grasp it (unless the end goal is to navigate artificial classroom conventions instead of truly developing communication skills). Most native speakers can’t even eloquently describe the rules they apply on a daily basis. Why is it a big red balloon but not a red big balloon? If prompted, I’d be astounded if an English speaker could tell you that the proper order of adjectives is indeed: 

Determiner > Quantity > Opinion > Size > Age > Shape > Color > Origin > Material > Purpose > Noun

yet somehow they manage to get it right intuitively.

And perhaps even more surprising, this incredible ability to infer what the proper order might be is not limited to native English speakers. Although some mistakes are often made in the beginning, even individuals who learn a second language after having an established internal linguistic system also seem to be able to grasp those nuances. Language instructors have been aware of this for a long time, and have been battling against standardized tests, which fail to properly assess communication skills.

It is also remarkable that with a finite amount of input, we are able to produce an infinite amount of output. Chances are you have (hopefully) never heard the sentence “The squirrel politely declined the job offer because the dress code required formal balloons on Fridays”, but you are still perfectly able to understand it (or in my case, produce it!). This is a key principle that Chomsky was getting at, and Syntactic Structures introduces a way to formalize the structure of natural language using a rule-based generative system. He believed in the existence of deep structures and surface structures – the former being essentially the core meaning of a sentence, as simplified as it can be, and the latter being the actual sentences heard or spoken. Of course, Transformational Generative Grammar doesn’t account for the impossibility of a squirrel politely declining a job offer (everybody knows they’re short-tempered and would probably do it passive-aggressively), but it focuses solely on grammaticality.

Of course, this is a high-level overview of a few of Chomsky’s ideas that stuck with me. There’s a lot more where this came from, and if you are interested in diving deeper into any of the topics mentioned, here are a few good places to start: