When "Pausing" LLM Development, Look to the Humanists
Humanities scholars (like those in Literature and History) have the training to deconstruct bias and power structures in language
A few weeks ago I speculated whether chatbots like Bing might be considered sentient in some way. There, I proposed the idea, which has been made over and over by critical theorists, that language is the raw material for the creation of personality. Basically, a personality does not express thoughts or feelings, but rather, patterns of thought and feelings are what bring a personality into being. Following this logic, a large-language model (LLM) might eventually accumulate something like a personality by spewing massive amounts of text.
This seems self-evident to a literary scholar like my self, who has spent her career excavating meaning from expressive word forms. But other perspectives on AI “consciousness” exist, which I’m going to discuss in response to the recent "Pause Giant AI Experiments" Open Letter. In Computational Linguistics, for example, the work of Emily Bender argues that so called "Artificial Intelligence" isn't intelligent at all, but just highly efficient at pattern matching. Bender has an intimate understanding of how computers process language, having spent her professional life studying languages and grammars with computational methods. In a recent tweet, she critiques the Open Letter for "dripping with #AIhype," like the belief that LLMs have "digital minds" or that AI is "too powerful." The real danger of LLMs, which she has been repeating since her famous paper on "Stochastic Parrots,” is that they concentrate power, perpetuate systemic discrimination and oppression, and proliferate disinformation.
According to Bender, is not that technology understands humans, but that humans misunderstand it. In an earlier paper, "Climbing Towards NLU," she and her co-author, Alexander Koller, explain that while LLMs may be adept at "learning" language patterns from processing large amounts of text data, they do not intuit intent. According to Bender and Koller, language meaning derives from a combination of expression and intent, a process summarized by the concise formula: M ⊆ E × I. Bender maintains that intent is not contained within word forms, but is something external, which can only be deduced or imagined in the mind of the interlocutor. Becuase it is external, intent will alway remains inaccessible to computers, who are constrained to a training process that consists of passively processing text. Studies in language acquisition have already proved such this idea to be problematic, such as those that test language learning from watching TV.1 Humans must be active participants in order to construct meaning.
For us, extrapolating intent is instinctual. We see meaning in everything. And as anybody who has taken a literature or history class knows, we especially see meaning in language. That being said, I’m not convinced that we should ascribe a similar logic about humans and learning toward computers. Doing so overlooks other possibilities, particularly for studying the ways that LLMs perpetuate cultural bias.
This is perhaps why scholar Ted Underwood, who explores computational approaches to studying literature, points out that "in historical disciplines, it is far from obvious that all meaning boils down to intentional communication between individuals." As any humanist scholar knows, are entire academic fields of study dedicated to different methods of critical analysis, such as psychoanalysis, to post-structuralism, to queer theory. Excavating meaning from the traces of history, from texts that might be severed from an explicit intention or known author, is something that humanists have been doing for centuries. Underwood suggests that LLMs might be used for comparative analysis, "not to mimic individual language understanding, but to represent specific cultural practices (like styles or expository templates) so they can be studied and creatively remixed."
I would revise Underwood’s emphasis here: LLMs could offer opportunities for studying how language encodes and perpetuates bias, racism, and xenophobia in general. However, the powers who train and distribute the LLMs are not terribly interested in studying how language engages with bias, despite their statements to the contrary. Rather, they are interested in developing products that will be attractive to everyday people and businesses, such as the numerous tools already being developed from GPT-4 technology. The problem is that the race for monetize the tech uncritically reproduces biases from training set. Models built to ingest as much data as possible, as quickly as possible, will reproduce only the dominant view.
That being said, if any group of people is equipped to deconstruct the ways that LLMs work to stifle minority experience, it is precisely people like Bender and Underwood, who have spent their careers studying how language creates and perpetuates power structures and social norms. It is the humanists, especially the ones in cultural and ethnic studies, who apply lenses from Queer, Black, Chicanx, Indigenous, and other minority perspectives as frameworks for analyzing cultural materials. If only a “pause” would divert support to the ones who are trained to do this necessary work.
Image credit: “Artificial Intelligence - Resembling Human Brain” by deepakiqlect is licensed under CC BY-SA 2.0.
Bender and Koller cite the following in their paper: Catherine E Snow, Anjo Arlman-Rupp, Yvonne Hassing, Jan Jobse, Jan Joosten, and Jan Vorster. 1976. Mothers’ speech in three social classes. Journal of Psycholinguistic Research, 5(1):1–20; Patricia K. Kuhl. 2007. Is speech learning ‘gated’ by the social brain? Developmental Science, 10(1):110–120.