Successful Patterns are the Best Guide to Information Design

I am very grateful to Jonatan Lundin for a lengthy conversation on the subject of topic patterns because it helped me to crystalize something important about the basis for the principles of EPPO information design and how they are derived.

Approaches based on psychology

Traditionally, theories of information design have been psychologically based. Researchers (usually academics) attempted to form a psychological theory about how we learn and then suggested information design approaches based on those theories. The success of such efforts has been mixed.

Market-based approaches

While I cite a number of studies in my work on Every Page is Page One, EPPO is not such a theory. Its roots are not in psychology or learning theory, but in the information marketplace.

Users have abandoned traditional sources of technical information in droves, heading for the Web, relying on Google, and frequently landing on certain high profile sites such as Wikipedia and StackOverflow.

Those sites are not popular because they are big; they are big because they are popular. The Web is an incredibly efficient market, which means that user’s genuine preferences drive success more rapidly and more ruthlessly than ever before. Thus mighty sites like MySpace can fall to startups like Facebook in a very short period. This means that we can look to the sites that work for examples of content patterns that work.

The Web as an Efficient Information Market

By an efficient market, I mean one that rapidly rewards genuine value as opposed to, say, traditions or an entrenched position. In an inefficient market, an established company can continue to command market share and make a profit even without delivering a high level of value. People are stuck with the product, even if they don’t like it, because it is hard to move, or because it is too hard for competitors to enter the market and compete.

The Web is, in this sense, a very efficient market. Indeed, it is so efficient that some studies indicate that it is in many ways easier to enter the market than it is to stay in it. Innovation comes through replacement and the innovator’s dilemma makes it hard for established companies to develop the next round of innovative products.

Efficient Markets the Best Usability Labs

An efficient market is the best usability lab in the world, particularly for content. Put products into that market and the market will tell you which are the most usable. The foundation of the lean startup movement is exactly this: launch the minimal viable product (MVP) and see how the market reacts. It is more efficient today to launch an MVP and see how the market reacts than it is to do extensive study, research, and design outside of the market.

As David Campbell, head of Microsoft Azure notes:

“Instead of having a debate informed by decades of experience around whether a customer would want A or B, we define a testable hypothesis, which we quickly try to validate.”

An efficient market tells you more, and tells you faster, than previous experience or academic knowledge. Getting to market quickly is therefore the optimal way to learn and grow.

There is a saying in the lean startup movement that if you are not embarrassed by the quality of the first version of your product, you waited too long to launch.

Not all Markets are Efficient

Not all markets are such efficient generators of information, of course. The old market for technical communication was very inefficient (meaning it did not quickly reward high quality or punish low quality). Companies wrote books, put them in boxes with the product, and sent them out to the market. The reader did not have easy access to other sources of information so if the manual was bad, they could not vote with their feet. They had to struggle on with the information they had.

Poor quality information might affect the success of the product (though poor customer service does not doom every product by any means), but it was hard to separate the effect of the documentation from all the other factors that might affect the success of the product. And in any case, by the time the product cycle had gone around and it was time to draw conclusions about what worked and what didn’t, it was hard to extract any feedback that would impact future documentation — hard to know what qualities made for good or bad documentation experiences.

With the market telling us so little, and telling it so slowly, the only way to approach improved doc quality was the academic approach: laboratory testing and psychological theory.

But Science is Hard Too

The problem is that such studies are expensive to conduct and often produce inconsistent results. For example, a survey of studies on the effectiveness of screenshots found differing results from different studies.

“Van der Meij (1998) reported experimental results suggesting that manuals containing screenshots speeded up execution of tasks up to 30%. However, other studies found either no effects (Gellevij et al., 1999) or even negative effects (Nowaczyk & James, 1993). A closer inspection of test designs and materials reveals another aspect that is important besides task complexity: To be effective, screenshots have to serve a specific function, and their design needs to support this function efficiently.”

In other words, sometimes they work and sometimes they don’t. The ones that do something useful work better than those that don’t. All of which is to say that “it depends” and that creating conditions under which such questions can be studied independently of all the things “it depends” on is well nigh impossible.

Tech Comm is not alone in these difficulties. The social sciences in general often have a hard time even replicating experimental results, and there is a real question about the validity of results broadly, as expressed by John P. A. Ioannidis in his essay, Why Most Published Research Findings Are False — something that should come as no surprise to anyone who has paid even passing attention to the endless contest of bad-for-you/good-for-you study conclusions concerning every foodstuff known to man.

Real science, in other words, is hard. Even for the pros, it is a long, and laborious process that is generally able to tease out durable conclusions only over multiple iterations of theory and experiment, often taking decades.

Combining Insights from the Market and the Lab

This is not to suggest that the academic work on improving content quality is without value. After all, I cite a great deal of it, particularly John Carroll’s work. But I value it far more for its ability to help explain successes and failures than its ability to predict them.

In other words, I prefer to look to the markets to discover what actually works and what doesn’t, and then look to the academic studies to see if they can help me understand why the successful pattern I see in the market actually works.

When it comes to Every Page is Page One and bottom-up information Architecture, the patterns, though very clear, are so contrary to traditional orthodoxies of information design that it is reassuring to have the results of so many studies — from the paradox of sensemaking to information foraging theory to the long tail — to help give us some reassurance that indeed these patterns do work.

Understanding Why a Pattern Works

There is more to this than reassurance, though. The problem with finding successful patterns to emulate is the same as that of finding workable theories to implement. It is figuring out if that pattern or that theory applies in the case you are looking at.

Studies can potentially help us get from “this pattern works” to “this pattern works in this way for these people under these conditions”, which is where we need to be to produce something useful. Having a better idea of why a pattern works makes it easier to figure out when it will work.

But it starts by observing successful patterns, and by emulating them to see if you are similarly successful. The market is the best usability lab, and in the case of content, the Web is the best and fastest usability lab we have ever had. One particularly welcome feature of the Web is that the success of a topic pattern on the Web is not tied to the success of any particular product. We can see the same patterns across content on many products.

I don’t want to give you the impression that I believe in the unalloyed wisdom or markets. Some market conditions produce quality information very quickly; some slowly or not at all. It has to do with how free consumers are to choose and how easy it is for new suppliers to enter the market. Where the consumer is free to choose and suppliers are free to enter, the market produces winners and losers very quickly. The Web today is such a market, particularly for content, and it is therefore a remarkable laboratory for rapidly testing information designs. We should take the maximum advantage.

One downside of the market is that it does not explain why something works, and we should be happy to have the academy to help fill in that gap for us. But the Web today tells us what content patterns work far more swiftly and reliably than the lab can.

Every Page is Page One is about User Behavior

I have always said that Every Page is Page One is not, first and foremost, a design principle, but an observation about reader behavior. The reason to write in an Every Page is Page One style is because people read that way even when you don’t write that way.

The Every Page is Page One principle that a topic should adhere to a well-defined topic pattern is based, primarily, on the observation that the most successful content on the Web, particularly the most successful technical content, tends to follow well-defined patterns that we can see emerging quite strikingly on platforms like Wikipedia.

Are there academic and psychological reasons to suppose that content that follows well-defined patterns would work better? Of course there are. But the market confirms that fact, and gives us practical models to follow whether we understand the psychology or not.

Optimizing Your Process to Learn from the Market

But besides these excellent reasons for adopting an Every Page is Page One approach, and for studying successful topic patterns, there is another more operational reason:

The best way to find out if your content is going to work for your intended audience is to publish it and measure the results. But this is only going to be effective if you can publish, measure, and update fast enough to learn which patterns work and which don’t and make practical use of the knowledge. An Every Page is Page One information design, especially if coupled with a bottom-up information architecture, helps you iterate your deliverables much faster, and therefore improve your topic patterns fast enough to make a real difference to your user’s success.