Sam Jackson makes it look easy, but Siri and her cohorts are still pretty dumb. If experience alone does not illustrate this to your satisfaction, a recent article in the MIT Technology Review, Tougher Turing Test Exposes Chatbots’ Stupidity, shows just how low their success rate is in understanding real language.
The Winograd Schema Challenge asks computers to make sense of sentences that are ambiguous but usually simple for humans to parse. Disambiguating Winograd Schema sentences requires some common-sense understanding. In the sentence “The city councilmen refused the demonstrators a permit because they feared violence,” it is logically unclear who the word “they” refers to, although humans understand because of the broader context.
The programs entered into the challenge were a little better than random at choosing the correct meaning of sentences. The best two entrants were correct 48 percent of the time, compared to 45 percent if the answers are chosen at random.
“Context” is the go to word to describe this class of recognition problem. It really does not tell us what “context” is.
We have always had a rather mechanical focus in our attempts to understand language. We think of it in terms of grammar and vocabulary. I would blame this on programming languages, which are absolutely made up of grammar and vocabulary, but our tendency to look at language this way far predates programming.
When applying the same grammar rules to the same vocabulary does not produce the same result in all cases, we appeal to context. That works in programming languages too, where we have the concept of namespaces. Namespaces are essentially a way of saying that a given word means one thing in this space and something else in another space. Namespace clarification is about knowing which space you are in: context.
But consider the very first example from the article, a phrase that Siri used to get wrong with potentially tragic consequences.
User: Siri, call me an ambulance.
Siri: Okay, from now on I’ll call you “an ambulance.”
How exactly do humans instantly and effortlessly tell the difference between these sentences?
Call me an ambulance.
Call me a romantic.
Call me Grandpa.
Call me later.
Context seems to me a wholly inadequate way to describe the difference and how we tell the difference. To me it is all about story. Or, to put it another way, if it is about context, then context is about story. And to put that another way, context is about story, not about namespaces.
“Call me later” does not mean what it means simply because “later” is a predicate of time, but because it is a shorthand for a common thing we do all the time.
Why does “call me a romantic” mean “ascribe the characteristic of romanticism to me,” and not “summon John Keats to my presence” while “call me an ambulance” means just the opposite? Because in each case, it refers to the story that makes sense. But that recognition does not depend on the rules of grammar or vocabulary, but on familiarity with those stories.
Much of humor, and puns in particular, depend on our ability to recognize the reference to many potential stories in the same phrases and to discriminate among them. The humor depends on the recognition that another more absurd story might as easily be the referent.
I know diddly about AI, so I have no idea why stories are hard for computers or whether AI researches are focusing too much on rules and not enough on stories. I do wonder if it may be hard to teach computers stories given that we learn most of ours through lived experience rather than through language. Language, it seems to me, only works as a shorthand for referring to the common stories of common lived experience and really does not do well at fully conveying an idea for which the hearer has no experience to draw on.
Can we teach computers our stories from linguistic input alone? Can we decode whatever it is in our brains that remembers our experiences and associates stories with them? Not a clue.
But seeing languages in terms of stories rather than rules matters for writers in their everyday work. There is no such thing as a plain statement of fact. We communicate by reference to shared stories and where the recipient does not share our stories they will either misunderstand or not understand.
There is no formal absence of ambiguity in absolute terms. Ambiguity is omnipresent in language and is resolved by the recognition of common stories. Even the smallest variation in personal experience can color one’s understanding of a communication.
We do not achieve clarity through absolute statement or through precise vocabularies (except in very limited and regulated circumstances like air traffic control or the operating room) but through the steady triangulation of stories that narrow the ground of common experience to within a sufficiently precise radius of meaning.
Writers recognize that we need to understand our audiences. It is perhaps less clear exactly what we have to understand about them. But it is this: we have to understand their stories.
But it is equally important that we understand ourselves in the same way. Most of our language is based on tacit stories. Stories from experience are tacitly known. We have never fully articulated them in language. But if successful communication depends on crossing the bridge between the stories you know and the stories I know, I need a more than tacit grasp of my own stories.
Successful writing is not the restatement of things learned from language, but the fresh statement of things learned from experience. It depends on the shrewd recognition of when we can rely on the tacit stories we share with our readers and when we must make the effort to make our tacit stories explicit, and the ability to do so in terms of the stories that our reader knows, and the words they know them by.
