And suddenly every tech comm and content strategy conference seems to be about getting your content ready for chatbots. Makes sense if you are a conference organizer. Chatbots are sexy and sex sells, even if the definition of sexy is a grey box with a speaker sitting on the counter.
But chatbots are not the future of technical communication. Here’s why:
Chatbots are stupid
No, I don’t mean that they are a stupid idea. I mean they are actually stupid. As in they are not very bright. As Will Knight writes in Tougher Turing Test Exposes Chatbots’ Stupidity in the MIT Technology Review, current AI does barely better than chance in deciphering the ambiguity in a sentence like: “The city councilmen refused the demonstrators a permit because they feared violence.” (Who feared the violence?) Human do this so easily we rarely even notice that the ambiguity exists. AI’s can’t.
As Brian Bergstein points out in The Great AI Paradox (also MIT Technology Review), an AI that is playing Go has no idea that it is playing Go. It is just analysing a statistical dataset. As Bergstein writes:
Patrick Winston, a professor of AI and computer science at MIT, says it would be more helpful to describe the developments of the past few years as having occurred in “computational statistics” rather than in AI. One of the leading researchers in the field, Yann LeCun, Facebook’s director of AI, said at a Future of Work conference at MIT in November that machines are far from having “the essence of intelligence.” That includes the ability to understand the physical world well enough to make predictions about basic aspects of it—to observe one thing and then use background knowledge to figure out what other things must also be true. Another way of saying this is that machines don’t have common sense.
Chatbots, in other words, may be great at ordering stuff from Amazon or telling you to put a coat on because the forecast says it is going to rain, but they are nowhere near ready to help you fix your technical problem.
But even if they were a lot smarter than they are, chatbots would still not be the future of technical communication.
Chatbots are a CLI
Chatbots are a command line interface. You ask them something. They reply (often stupidly). That is what a command line interface does. In fact, we have had chatbots you can type at for a long time. ELIZA, a chatbot created in the 1960’s at MIT Artificial Intelligence Lab could act as a Rogerian psychotherapist. Uttering comforting platitudes to the broken hearted is not the height of intelligence. Any sympathetic school child can do it. Solving complex technical problems is much more complicated because the problem area is much more diverse. Putting a voice interface on the AI isn’t going to change that. Command line interfaces, whether visual or verbal, still have the same problem they have always had: they don’t support discovery and exploration.
Since the 70s we have has text adventure games like Colossal Cave Adventure where you discover your environment with conversations like this:
YOU ARE STANDING AT THE END OF A ROAD BEFORE A SMALL BRICK BUILDING.
AROUND YOU IS A FOREST. A SMALL STREAM FLOWS OUT OF THE BUILDING AND
DOWN A GULLY.
YOU ARE IN A VALLEY IN THE FOREST BESIDE A STREAM TUMBLING ALONG A
Would you prefer this interaction over a video game that actually shows you the forest and the stream tumbling along a rocky bed? (Or, you know, going outside and actually seeing a forest and a stream tumbling along a rocky bed?) It may have a particular kind of retro charm, but for practical purposes it is incredibly clumsy and laborious.
And the problem here is not one of how smart the AI is. It is the clumsy, time consuming, non-discoverable, hard to explore nature of the interface that is the heart of the problem. Making the AI smarter isn’t going to make the interface any more appealing.
Now admittedly, this is not a lot different from talking to technical support on the phone. The difference is that technical support is still (mostly) staffed by human beings who have that common sense, that “ability to understand the physical world well enough to make predictions about basic aspects of it—to observe one thing and then use background knowledge to figure out what other things must also be true” that AIs just don’t have yet, and that some speculate they may never have.
But the thing is, talking to tech support is not exactly technical communication nirvana. In fact, tales of tech support are predominantly tales of frustration on both sides of the phone. And what does tech support do if you ask them to help you with a truly complex problem? Chances are that they send you documentation and ask you to call back if you get stuck. And what to they do when they come across a common problem: they write a knowledge base article about it.
Some of that, of course, is about saving money, as tech support people have to be paid. With AIs, you could theoretically stay on the line for hours at minimal cost to the provider. But how many people are going to want to stay on line for hours with a chatbot? Only the lonely. Certainly not those in a hurry to get a job done.
But there are lots of problems that you would not even think of trying to solve in a conversation with tech support. Sometimes if you are going to ask an expert, you need the expert on site where they can see your work and watch what you are doing. Which brings us to the next problem with chatbots.
Chatbots are blind
Another limit of the chatbot interface is revealed by the text adventure interface. Not only can the chat bot not show you anything, it can’t see you do anything either. You have to tell it what you do.
An expert can look at your work and tell you what is wrong. A coach can watch you execute a maneuver and critique your form. A chat bot can’t see anything. Even if it could, the ability of the AI to make sense of what it is seeing and put it in the context of the user’s task just isn’t there yet. So you have to describe your problem to the chatbot. You have to turn your problem into words.
But turning your problem into words is difficult. It requires you to understand and articulate the problem, and by the time you can understand a problem well enough to articulate it, your are well on your way to solving it. The primary virtue of an expert is that they can look at what you are doing and spot the flaw that you cannot see. You know what result you are not getting. The expert spots the one thing in the hundreds of parts and pieces you have assembled that is on backwards.
A chat bot can’t watch you work. It can’t look at the work you have done. And you can’t tell it what the problem is because you cannot see the problem, only the result of the problem.
But even if your chatbot grew up and became a robot with the vision and the common sense and the specialized knowledge and experience to watch you, to examine your work, and to spot the flaw in it the way a human expert would, it still would not be the future of technical communication.
Why not? Setting aside the fact that if you had a robot with those capabilities you really would not need to learn to do the task yourself, the real problem is that there is only so much that experts can do for us. In the end, learning is about rearranging our own mental furniture, finding our way through the thickets of our own minds. The expert can help us enormously at certain key junctures in that process, but most of it we simply have to do for ourselves. Content can certainly be a huge help to us through the processes, but it has to be a type of content that is most amenable to the trial and error, the exploration, and the intuitive leaps of recognition and synthesis that are fundamental to that journey, and that is text.
Chatbots don’t support wayfinding
Most of learning is wayfinding. The iterative process of refining our mental models until they fit the world as we are discovering it and trying to manipulate it requires us to range broadly and often erratically across a large body of information. John Carroll’s work that led to the publication of The Nurnberg Funnel showed that different people traverse texts in different ways driven by the particulars of their individual task and background.
No simple, comprehensive, logical treatment of the paradox of sense-making is possible. The tension between personally meaningful interaction and guidance by a structured curriculum entails a priori limitations on how much we can ever accelerate learning.
Users must forage for information as they forage for insight while attempting to hack through the brambles of their own preconceptions. By far the easiest medium to do this foraging in is text. Nothing else lets you speed up and slow down, go straight or turn left with anything like the same ease. There are, to be sure, ancillary media that can play a valuable role in our foraging: maps, graphics, animations, etc. But it is text that leads us to them, and text that leads us on.
Text, particularly hypertext, excels in these areas. Voice interfaces suck at this. The eye can pick a relevant phrase out of a scanned text at considerable speed. A speeded up audio stream quickly becomes a high pitched babble. And the eye can speed up and slow down of its own accord, whereas the hand or the voice has to give commands to the audio stream. The eye can flip through a stack of paper or a list of search results in seconds. Navigating the same volume of audio material would take far longer and require far more interactions. And where the eye can command the hand to follow a link almost without thinking, embedding hypertext in voice just isn’t feasible. Audio is a fixed pace linear medium, yet the task of foraging for information and understanding is one that demands the ability to change pace and direction at will. Again, it is not the intelligence of the AI that is crucial here, but the nature of the interface and the nature of the task.
Even AI can’t build a Nurnberg Funnel
But what if the AI were to get so smart that it could avoid the need for you to do all this discovery? That certainly seems to be the hope that some of the current boosters of AIs in content strategy seem to be hinting at: the ultimate expression of delivering the right content in the right format to the right person at the right time. But that supposes that (all other issues being solved) we could know, with perfect certainty, what content is needed at a particular moment in the user’s journey to understanding. But we can never know that, because each user’s journey is unique and it is occurring largely in the privacy of their own skull.
Much of the wayfinding you have to do to learn something is navigating your own prejudices and presumptions, your own guesses that you think are facts. There is always some destruction to do in learning before construction can begin. Indeed, the destruction is the hard part, and it is all in your head and it is all unique to you. No human teacher and no AI is privy to that landscape. This is one of the most fundamental of human facts. We are born alone. We die alone. And we learn alone.
The notion seems to be that AI will remove the need for wayfinding by always being one step ahead of you. This is something no human teacher has ever achieved consistently, and there is a very good reason for this. It is the Nurnberg funnel, and there can be no Nurnberg Funnel. As John Carroll wrote:
Users, no less than instructional designers, are searching for the Nurnberg Funnel, for a solution to the paradox of sense-making. But there is not and never was a Nurnberg Funnel. There is no instantaneous step across the paradox of sense-making.
There is no instantaneous step across the paradox of sense-making. There is only foraging for information and understanding across a field of information and experience. And the form of information most suited to creating that navigable, foragable field of information is text, the past, present, and future heart of technical communication.