Language Models and Meaning

Language Models can only understand language if they have a theory of meaning

Every theory of meaning or semantics will, in some form, link language to the world. Classical theories developed by philosophers such as Frege and Russell do so directly by offering theories of reference that link linguistic expressions directly to objects and properties in the world. A different school of thought sees meaning as rooted in social practice, linking language to a broader set of social behaviors that mediate its relation to the world. And a third approach ties language to the mental domain, connecting expressions to thoughts or concepts that themselves refer to the world. These approaches take many forms, overlap in various ways, and trade off explanatory priorities. But they all share a commitment of tying meaning to something extralinguistic—whether in the world, in thought, or in social activity. And it is precisely these links that appear to be absent in the case of large language models, given the nature of their training.

A language model like GPT is trained exclusively on textual patterns. Its only input is a stream of words, and its learning objective is to predict which word is most likely to follow a given context. The resulting model captures intricate regularities across language, forming powerful statistical representations of word sequences, syntactic patterns, and pragmatic conventions. But these regularities are ultimately just patterns between words and words alone. There is no opportunity in this training process for the model to link its internal representations to extralinguistic entities—no connection to people, objects, events, social cues, or intentional states. The learned structure remains entirely within the space of language. This absence of grounding entails that a language model is unable to support an essential component of a standard theory of meaning.

While the lack of grounding severs connections between word and object, it doesn't sever the semantic connection between words. A language model learns a great deal of linguistic structure, linguistic form and it is reasonable to suppose that this linguistic form could capture this internal semantics. The suggestion is plausible because you see this understanding in a language models ability to summarize, extend or rewrite a passage of text. And you see it even more clearly in its ability to translate not just between languages, but dialects and idioms. It is able to do so because it has a grasp of the meanings of the words and how they relate to one another. And it is not just that this semantic expertise is evident in a language model's performance, but it is exactly what is expected of a successful, properly trained language model.

The stance is compelling: the very nature of internal semantics appears to sidestep the grounding objection. But a closer look reveals its limitations. Much of the compelling evidence for this kind of internal semantics depends on our interaction with the model in a familiar language, one we already understand. In these contexts, we attribute structure and coherence to the model’s outputs because we interpret them from within a shared semantic frame. But if we revisit the earlier thought experiment—in which a subject learns a language only by predicting next words, without any connection to the world or prior semantic knowledge—the evidence you want to reveal doesn't find a purchase. Because you don't understand the language you don't know when you are asking for a summary, elaboration or a rewrite, and exactly the same point goes for any form of translation. You don't know when you are asking a question, making a statement, agreeing to a contract or simply greeting. The thought experiment underscores how completely in the black you are.

This difficulty can be sharpened with a common distinction in the philosophy of language. Suppose the language in question contains statements like “A scalpel is a surgical instrument” and “A scalpel is sharp.” The first is considered analytic—true in virtue of meanings alone—while the second is contingent, true due to facts about the world. In principle, a language model trained entirely on text might reproduce both statements correctly. But can it distinguish between them? Without access to extralinguistic context, the model treats both as highly probable sequences in appropriate contexts, but has no mechanism for classifying one as meaning-constitutive and the other as world-dependent. And even more fundamentally, it has no way to identify _which_ statements fall into either category. The sentence “A scalpel is a surgical instrument” appears in its training data not as a rule or definition, but simply as a sequence of tokens with a statistical profile.

The same applies across the board. There are thousands of true statements one could make about scalpels—some analytic, some empirical. The model can generate them because they are statistically likely in context. But their truth conditions do not enter into its reasoning. It outputs “A scalpel is sharp” not because it believes or infers that this is so, but because “is sharp” is a likely continuation given the prompt. In a different context, it may offer “is a medical instrument” or “is sterile,” all generated through the same probabilistic mechanism. The distinction between empirical fact and conceptual truth plays no role in the model’s operation. From the model’s perspective, all these outputs are equivalent: high-probability continuations shaped by past co-occurrence statistics.

What we observe, then, is a system that maps one sequence of words to another with extraordinary success. Its outputs often align with truth, and sometimes with analyticity. But it does not track either as such. It does not know which of its statements depend on how the world is, and which depend only on linguistic convention. It has no meta-linguistic awareness of these categories, because it lacks the concepts they presuppose. The claim is not that it performs poorly, but that its performance lacks the conceptual structure we normally associate with understanding. The behavior is impressive, but the basis for interpretation remains missing. To be clear, the language model does possess a rich structural understanding of language, structure which supports fluency in the language; however, this is not semantic structure that can support an internal semantics.

We now now quickly led to some strong conclusions. A language model may exhibit fluency, generate coherent and even insightful text, and display patterns we associate with intelligence. But if it lacks any framework—whether conceptual, perceptual, or social—for distinguishing the kinds of semantic roles that underlie language use, then it lacks a _theory of meaning_. But if it lacks a theory of meaning for a language then it fundamentally does not understand the language, no matter what its overt fluency appears to be.

But if a language model does not have an understanding of language then it cannot reason or understand the way humans do because language lies at the core of a human thought. Language is what separates man from all other species. It was the emergence of language among _homo sapiens_ 50 to 100 thousand years ago that allow the species to dominate the entire planet. Historically this has been uncontroversial: from Aristotle to Chomsky, language has been seen as the key to human intelligence. Aristotle argued that _logos_, which is speech and reason, is unique to humans;  Rene Descartes thought animals lacked reason because they lacked language; Wilhelm von Humboldt saw language as the _formative organ of thought_; and Noam Chomsky insisted that the language as a uniquely human faculty. If a language model does not understand language then whatever it is doing should not be compared to human reason and thought.

I find myself in a genuine quandary. The conclusions I’ve drawn seem to place me in the same camp as people like Melanie Mitchell and Gary Marcus, who are sharply critical of language models and insist they lack real understanding, reasoning, or intelligence. But I don’t share their overall stance. I think these models are astonishing—capable of producing fluent, coherent, and often insightful responses at a scale and depth never before seen. That kind of performance demands explanation, not dismissal. But at the moment I have no explanation.

I actually think this is a good place to be. The thought experiment doesn't give us final answers, but it isolates a real and important difference between human and artificial intelligence. The task is not to force them into our conceptual categories, but to understand them on their own terms. That may mean rethinking what we mean by language and intelligence or what we think a language model actually is. But for the moment I have nothing.