Hierarchy as a form of content organization is dying. A major milestone — I want to say tombstone — in its demise is the shutdown of the Yahoo directory, which will occur at the end of the year according to an article in Ars Technica, Yahoo killing off Yahoo after 20 years of hierarchical organization. (Actually it seems to be offline already.)
As the article observes, a hierarchical directory made some sense when Yahoo was created:
In the early days of the Web, these categorized, human-curated Web listings were all the rage. Search engines existed, but rapidly became notorious for their poor result quality. On a Web that was substantially smaller than the one we enjoy today, directories were a useful alternative way of finding sites of interest.
But as search engines have grown more capable, and the Web has grown exponentially larger, they just don’t work anymore:
As the Web grew, directories became less useful—there was no way they could ever hope to be exhaustive—and Google, in particular, made search engines useful. The directory fell out of fashion. … it clearly no longer captures the interests or imaginations of Internet users today.
The phrase the article chooses is particularly telling: “The directory fell out of fashion”. Search, as well as various forms of social curation, have risen in popularity because of the simple impossibility of maintaining or navigating a directory of the whole Web. But in doing so, they have driven the very idea of such a directory out of fashion, even at a scale where it might still be effective.
The reader is search-dominant
Readers are becoming more and more search-dominant. Their directory-using skills are eroding. This means increasingly that you cannot improve information access by building a better directory because directories themselves are out of fashion. Improving your TOC won’t make it easier for someone to find information if it never occurs to them to use the TOC.
It isn’t just search and social curation that is hammering nails in the coffin of hierarchy. Other forms of categorization are hastening its demise as well. There is faceted navigation, for example, in which multiple different criteria are presented as peers and users can drill down according to whatever criteria they choose, rather than according to a fixed hierarchy.
One thing search, social curation, and faceted navigation all have in common is that they are dynamic: they require a active algorithm, a sufficiently large datastore, and a means to connect to both. Hierarchy, by contrast can be implemented statically. Hierarchy’s dominant position pre-Internet is thus in part due to its ability to be implemented in bricks and mortar, in shelves and bins, and on paper. A networked digital world has destroyed that technological advantage, and is busy ripping apart the social consensus about content organization that was built on it.
A scale that can only be handled by filtering
Another factor in the ongoing demise of hierarchy, as the Ars Technica article notes, is the sheer scale of the Web — not only how truly vast it is, but how rapidly it changes. A content set this huge and this volatile can never be mapped. You might as well try to capture the topography of the rolling ocean. The Web can never be comprehended. You can never be sure that you have pinpointed the best it has to offer. The whole approach to discovery has to be different: to use the Web you must filter it.
To use David Weinberger’s words (which I quote so often), the new default is “Include it all. Filter it afterward.” In other words, we make no attempt to constrain the initial set of content that we consult — we consult everything and then we apply filters to it, usually in the form of search terms, which rely on the search engine’s ranking algorithm and everything the search engine knows about us, in order to filter forward the most likely results.
This works far better than trying to navigate a hierarchy of the whole — even if one could be created and maintained, which it can’t. And much of the time it works better than navigating the hierarchy of a small portion of the whole, since the main problem for the reader is finding which small portion contains the information they need. There are multiple ways of solving this problem, but searching is the first and most obvious — and the one that adapts best to changes in the content.
Navigating bottom-up
One of the things worth noting about the difference between the Yahoo directory and Google search is that the directory indexed sites whereas Google indexes pages. The Yahoo directory was a hierarchy of hierarchies; Google search is a filter of pages in which the site plays only a partial weighting role through its reputation. Yahoo attempted to take you to the top and help you work your way down. Google takes you straight to the bottom and leaves you to work your way up or sideways.
The reader’s new default
These are profoundly different information seeking paradigms. There is no a-priori way for a reader to know which will be the most successful for any given enquiry, therefore their natural tendency will be to turn always to the one that is more often successful and that is intellectually less demanding: search. Thus people increasingly turn to search even in information sets that are still small enough to be navigated hierarchically.
People encounter content bottom up, and we need to meet their needs with a bottom-up information architecture.
Not dead, just resting
This is not to say that hierarchy is entirely dead. This is not that kind of death. As with so many other technical and economic deaths, this is not the death of extinction, but the death of insignificance, the particularly harrowing death of falling out of fashion. The death of smokestack industries does not mean that no one makes refrigerators anymore. It simply means that the making of refrigerators is not what drives the economy today. Hierarchy can still work for some forms of content organization, but it is not what drives information finding today.
The death of hierarchy means that it has lost its dominant and normative position as the gold standard of content organization. It still has notable secondary uses. Computer file systems continue to be hierarchical, for instance. Even the URL addressing system of the Web is nominally hierarchical, though it does not consistently express any hierarchical relationship of the content that URLs point to. Its role today is utilitarian, not normative. It does not dominate the patterns of thought and action of the modern information seeker.
Writing in a post-hierarchical world
The difficulty for writers is that we are struggling to figure out how to work in a post-hierarchy world. In a recent post on the Techwhr-l discussion list, Getting users to RTFM, David Tingley writes:
My tech support colleagues continually get calls from customers for which the response is RTFM. The manuals do contain all the information needed, but it seems the customers would rather pick up the phone to tech support. We have been brainstorming ways of making it less intimidating for the customer to find information in the manuals. We deliver pdf, fully indexed, cross referenced and with a comprehensive logical TOC.
He signs the message:
David (who never reads manuals unless it is to critique them!)
In the comments that follow, other writers confess that they don’t read the manual either. Even those who claim they do, seem conscious that doing so makes them an anomaly. We continue to make the old hierarchical artefacts even though we ourselves have become post-hierarchical in our information-seeking habits.
David’s company is doing everything right by the old rules of the age of hierarchy. They “deliver pdf, fully indexed, cross referenced and with a comprehensive logical TOC”. And nobody uses it. Because hierarchy is dead.
There are doubtless some customers who will always call customer support as a first resort. There is not a lot we can do about them. But there are also a large number whose first resort will be to do a Google search. Locating and using a hierarchically organized manual, on the other hand, will usually be their last resort, if they resort to it at all.
Improving the manual, therefore, will do nothing to deflect tech support calls. There is nothing you can do to persuade people to abandon their information seeking defaults. There isn’t even a way to get them to be aware of your efforts to do so. As long as you stay in the hierarchical world, much of your intended audience is simply blind to you.
The opportunity to deflect support calls now lies in the ability to address the user’s needs at the time they do their Google search. To do that, you must first put your content where Google can index it. You must then design it so that it works for the reader when they encounter it bottom up. Every page must work as page one. It must be self-contained, establish its context, and link richly to pages on associated subjects.
Hmmm… I don’t think Google will ever index my embedded documentation. And I think that’s the primary place to put information about a software product — in the GUI. So really, the GUI is page one, and every other page is page 2 or greater. At least that’s how it is in my world.
If you want to talk about breaking old paradigms, then the paradigm of turning to the documentation, any documentation, that is not part of the product’s surface is what we need to break. Certainly, for the first tier of docs, that must be true. In other words, give the user content before she asks for it, not after. And if that documentation isn’t sufficient, you have every opportunity to add in the scent of information right there, on the surface of the product (as in, links).
The thing about modern, virtual products is that the surface is mutable. And the product itself is primarily information. Why would you not treat that like a page of documentation?
But to stick to the point… Product surfaces often exhibit hierarchies. They certainly exhibit controlled organization. So within the domain of a product, I’m not ready to lay the wreath on hierarchy.
Thanks for the comment, cud.
I don’t disagree about the virtue of embedded documentation in some circumstances. But, as I pointed out in my last post, the interface is itself documentation (/2014/09/22/the-interface-is-technical-communication/), so embedding docs that merely repeat what the interface is saying is pointless. And neither the interface nor embedded documentation can answer any question that the user has before they have located to correct part of the interface. They can’t, therefore, answer questions about how to solve a particular business problem using the tool. They can’t even answer questions about which part of the tool to use. They can only answer questions about how to use the part of the tool the user is currently looking at.
Since the interface itself is designed to answer those types of questions, this actually leaves a fairly limited — though often important — role for embedded documentation to play.
And, as you suggest, the distinction between interface and documentation is pretty fuzzy in these circumstances. When is it embedded documentation, and when it is just a text-based interface? And does any distinction we make have more to do with the org chart of the vendor than with the function of the interface?
Product surfaces do indeed often exhibit hierarchies, though as products become more Web-like, there is also a tendency for them to exhibit hypertext characteristics, and, as you say, more mutable surfaces. But hierarchies can work quite well on the small scale — and even on a larger scale if the principle of organization is uncomplicated. Hierarchies can still be utilitarian for many purposes. That is why the wreath I am laying on on the tomb of hierarchy as the defining principle of content organization, not on all uses of hierarchy.
Perhaps the concept of ‘page’, being mutable and evocative of a two-dimension surface needs to change to something of a different ‘shape’. I couldn’t agree more that hierachies are indeed dying; the question is: what do we replace them with?
The answer, I believe, is multi-dimensional graphs, starting with faceted vocabularies.
Thanks for the comment, John.
As to what to replace hierarchies with, from a consumption point of view that answer is that they have been replaced by search and hypertext (a very necessary and underappreciated pairing). Hypetexts are, of course, a form of multi-dimensional graph.
On the creation side, though, we also need something to replace them with and multi-dimensional graphs and faceded vocabularies are indeed strong candidates. The SPFE project (http://SPFE.info) is intended to bring these kinds of tools to the creation side through metadata-drive content organization and linking.
The enterprise is not the internet, though, and while I agree that the combinations and permutations are infinite ‘out there’, inside an organization the elements (you mentioned the words ‘bricks and motar’ can be identified, classified and organized in several useful ways without the need for algorithms or even (technically) indexes. Filters are naturally faceted and the model is technology agnostic (although it prefers graphs) and language neutral.
The idea is based on the six universal interogatives and follows the same ‘declarative –>associative–>narrative’ model that application developers, map makers and story tellers follow to present there respective “UI” offerings.
I’m working with a couple of DITA gurus to begin to demonstrate the value of what I’ve coined as “Dimensional Enterprise Information Management”.
Stay tuned!
I found this passage really interesting:
Are you saying that flat files are on the demise in favor of database structures that allow for dynamic querying and filtering of content?
DITA’s main paradigm (and many other structured authoring systems) rely on flat files, for the most part. This makes it easier to upload the content onto a server, thus bypassing authentication issues. I found that the main obstacle to implementing a platform like WordPress for help content is that I couldn’t authenticate users except by custom-developing a single sign on mechanism to another system (e.g., Salesforce) , which they were already using to authenticate to access their content. Flat files uploaded and delivered via a third-party (e.g., Salesforce) interface bypassed the authentication issues.
Do you want to expand on whether flat files will die out in favor of database structures? There is a resurgence of simple flat file system content management systems (like pico, jekly, wintersmith, docpad, kirby, docco) that try to sidestep the larger database frameworks that developers largely find unnecessarily for delivering simple content. Would you say these simple flat file CMSs are a trend that will be short-lived?
Finally, I just wanted to say that I agree that TOCs fail for large amounts of content. But TOCs work fine and are probably superior to no TOC when you have a small set of information (e.g., 10-20 pages). In other words, mini-TOCs work pretty well, in my opinion. I’m always wondering if you’re making an apples to oranges argument when you compare the Internet to help documentation.
Thanks for the comment, Tom.
No, I don’t see this trend leading to a demise of flat file content, simply because the filter mechanisms that people are using today are external to the content itself. In fact, I would suggest that the trend you note of a resurgence of flat file sites may be precisely because people are realizing (tacitly or otherwise) that people are navigating using external filters rather than internal navigation mechanisms.
If you come to see your site as contributing a collection of resources to the Web, rather than acting as an independent stand-alone publication, then a grand organizing principle, and the mechanisms to create it, become of much lesser importance.
If your content is behind a login, of course, you are hiding it from many people’s preferred method of information seeking (and thus losing one chance to deflect calls from tech support).
But even if people do login to your site, their fashion tastes in information seeking don’t change when they pass the threshold. They are still going to seek information in the same way. On general principle, then, I would still suggest flat file content, richly linked, and accessible to the best search engine you can find, as the best default.
Particular circumstances may, of course, indicate a different approach, but always remember that if you don’t give people a search box, you are asking them to learn you navigation system before they can get the information they want. There needs to be a really good reason to put that additional hurdle in the reader’s way. (And if you do give them a search box, that is what they will use.)
Since the reader is increasingly using external filters to enter your content set, this moves the emphasis for the writer to local bottom-up navigation to help the reader move around once they have landed on whatever page has become their page one. This means hypertext, which means links. We do need systems, therefore, that can help us build and manage links.
Here is where databases may come in useful, though there are certainly good solutions at the flat file level as well. The big difference here is not so much capability as the cost of integrating new content. A database can often avoid the need to rebuild all the content when a new item is added (and needs to be linked to). A flat file system generally needs to rebuild everything, which may or may not be a significant issue.
I agree about mini TOCs. They can be very valuable. But they exist at the bottom. They are not a grand organizing principle, but an item of local navigation. In fact, I would characterize them as a type of list, and lists are very important in hypertext systems. Hierarchies still make good foot soldiers. They just don’t make good generals anymore.
The great thing about local lists is that they work equally well if the reader is directed straight to them, or if they stumble on them through search or by following a link.
In fact, the ability to generate lists automatically is one of the things I consider basic in a good bottom-up information architecture.
Hi Mark, another amazing article!
Another aspect that might be worth mentioning is that beyond social curation, the expectation of social interaction too is killing the traditional content hierarchy. When we see this interaction everywhere from WordPress blog comments to Wikipedia article “Talk tab” discussions to Amazon product reviews, it’s natural to expect it elsewhere.
Sadly, even if a topic hierarchy provides commenting functionality, I rarely see folks using it, at least not to the frequency of tweets or likes. I’m not sure if this inactivity is related to the sense of indifferent “static” content, or if it’s the comparative lack of “cohesion” among hierarchical topics (i.e. it’s easier to reply to a self-contained article than to fragmented topics with low cohesion), or if it’s a combination of both, but something is definitely missing.
In fact, something else that might be overlooked is that the social comments, discussions, and reviews themselves can add a content “richness” and fill in the gaps or touch related ideas that weren’t necessarily contained in the original topic or article. Just as an Amazon product page with a dozen different reviews and perspectives can yield much more insight than the same page with none.
Thanks for the comment, Jay. You raise an interesting point. Certainly the social aspect of a site like StackOverflow is vital to its success, and to its organization. Reader response is a vital part of the way the Web is filtered. Google does it implicitly. StackOverflow does it explicitly. Readers clearly look for it and appreciate it.
One of the points I often make about how the Web has changed our informations seeking habits is how it has changed our attitude to authority. It used to be you the way you sought information was to first find and authority, then ask your question. Now the approach is first ask your question, then consider the authority of the answer. And authority here is socially mediated. You trust the guy on StackOverflow with the high reputation more than the company’ official word on the subject.
I would love to know how much social interaction with content corresponds to how EPPO it is vs how hierarchical it is. I suspect that EPPO content is more commentable, because it is complete and sets a context. It is clear what you are commenting on. I would love to have a chance to examine some examples to see if that is true in practice.
I definitely agree that comments can add richness and fill in gaps in content. I don’t have to look any further than this blog — indeed, than this topic — to see evidence of that. And I think it changes how we write too. We don’t have to write “the final word” on a subject. We just have to start a discussion.
Hierarchy, of course, is about having the final word on the organization of content (and thereby the final world on its significance). A post-hierarchical world says that the author does not get the final word on the organization or the significance of the content.
A post-hierarchical world says that the author does not get the final word on the organization or the significance of the content.
Great comment and one that also suggests that faceted search can start where the user wants, not where someone ‘knows’ she has to.
Absolutely. But faceted search only works if the user understands the facets. Navigation has to be multi-modal today.
What if there were only six (facets) based on the structure of natural language, and designed to answer the basic business questions in an enterprise? 😀
Care to share an example?
My go to case for faceted navigation is used car sites such as autocatch.com. They actually have quite a number of facets, but each is concrete and familiar to the average car shopper. Plus the vocabulary of each facet is well known and unambiguous. Where these conditions are true, faceted search works really well.
What undoes faceted search is ambiguity and unfamiliar or imprecise terminology on the user part. (We can always drive ambiguity and imprecision out of our view of a system, but not out of our users heads.)
By the way, is the project you are working on in any way related to Jonatan Lundin’s SeeSam project?
Taking autocatch,ca as a first example (always good to start with something familiar) here are the key facets:
Physical Asset – a Thing that exists in real time/space – in this case the actual vehicle (represented by a picture)
Place – in a naturally occurring subsumption hierarchy; e.g. “Canada”, “Alberta”, “Calgary”. More Places available on request
Status – “Used” and “New”
Concept – The make and model names are persistent across many years, and qualify as values in this class
Span – A Year value represents a span of 12 months; A Price Range is also a span between two points.
Relative Position – “Over” / “Under”
Task – “Browse” is not associated with the Primary Asset (the vehicle) but it demonstrates how the model can be universally applied.
Organization – This one is not explicitly represented, but the Concept of ‘Make’ could be aggregated by the Company (“General Motors”, “Ford”, “Nissan”, etc.) that acts in the Role of Manufacturer.
Role – “Buyer”, “Seller”, (I already mentioned “Manufacturer”) “Shipper”, “Dealer” are all valid values.
Example number 2: When I was working with Lu Hall and Karen Lowe on a new production accounting system for oil and gas, they authored their topics in DITA. One of the requirements was to get users to relevant content in two clicks.
The interface had three questions ‘built in” that users could ask to get to in-line Help:
What is…? Accessed by cllcking on the question mark
How do I …? By typing an imperative (e.g.: Add, Move, Post, etc) into the search box.
Tell me about…? (Which isn’t really a question…but) By typing an accounting, process, or applicaiton term or phrase into the search box.
The results sets cam back organized along the same facets as the questions, but also included ways to get at what we called ‘Intersecting facets: Location, Asset, Function, etc) to broaden their search to include company-specific terminology.
The application Help could also link to training videos and a corporate wiki.
I’ve also performed the same analysis for health care without adjusting the facets at all. (To answer your question about the SeeSam project, no I’m not familiar with Jonatan’s work.)
Thanks for opening your blog to me, Mark. I appreciate the opportunity.
Thanks for the details, John.
I think you might find Jonatan’s work interesting. I have the same question for you that I have for him, though: what do you see as the virtue of generic facets over domain specific ones?
For instance, while your translation of the autocatch facets from the particular to the general seems reasonable, I can’t imagine that autocatch would be made more useful by changing the interface to ask customers for place, concept, and organization rather than make, model, and color.
Great question.
Generic facets are foundational not prescriptive (in the UI) so theoretically carcatch could use the generic labels, but any label they choose to use fits into one of the six foundational facets. The approach is the same as object oriented application development: a programmer would likely not use the Name of a Class, but an Instance of a Class as an object name in the interface.
In DITA, for example, if I declare Title as an Element, the actual Title I use is a member of that Element Class.
The principal is: the classification protocol is not negotiable. The values you run against the protocol is only limited by the purpose of the end product.
Hi John and Mark, fascinating discussion! But let me play the Devil’s Advocate. While I might be less familiar with the term “facets”, I’m assuming that they’re analogous or synonymous with filters. If so, and a visitor can start anywhere in the faceted hierarchy, couldn’t she also bypass the facets altogether by starting with a Google search in the first place?
For example, if I perform a Google search for “autocatch ottawa nissan juke”, the first result takes me directly to the same destination as if I drilled down into the Autocatch faceted hierarchy. While it’s true that these search terms act as implicit filters, I never think of them as facets or filters. So, compared to a Google search or even the site-level Autocatch search, wouldn’t a faceted hierarchy still have the same weaknesses as a TOC-based or DITA-map hierarchy? I apologize if I misunderstood anything. But thanks for listening!
Another good question, Jay.
Think of facets as sides of a cube with your search results somewhere in the middle. Now think of Acronyms, Abbreviations, Words and Names being assigned to each facet, like x.y. x or latitude, longitude and elevation. Now think of the query you typed into Google as a set of coordinates, a la GPS.
You can filter the result set by (sometimes) different refiners to your heart’s content.
No hierarchy required!
A-ha, great metaphor!
So in the case of Autocatch, their site doesn’t really need its faceted navigation. If they wanted to, they could rely solely on its site-level search. Or if they wanted to be really extreme, they could strip it down and rely solely on Google search, haha No site-level search or hierarchy required! Thanks John!
Hi Jay.
Good question. First, let me clear something up. A faceted navigation isn’t a hierarchy. A hierarchy subordinates one facet to another, so a hierarchical car site might be organized like this:
make > model > year > style > color >transmission
But if I want to shop for cars with manual transmissions, I will have a hard time because they are scattered all over the hierarchy and there is no easy way to get a list of just manual transmission vehicles.
A faceted navigation makes each of the facets peers and let you choose to narrow down the list by any facets in any order. So if you want only manual transmission cars, you can set the transmission facet to “manual” and the list will only include manual transmissions.
The great virtue of faceted classification is that it does not impose an order on the user’s search. The problem with them is that they rely on the facets being clear and unambiguous and the user recognizing the facets and knowing the vocabulary of each facet. That is the case for cars. It is often not the case for documentation.
And you are quite correct that all three are cases of facet-based navigation. A hierarchy is made up of facets in a fixed order. A faceted navigation is made up of clearly defined facets that can be selected in any order. A search engine takes search terms as facets of a fuzzy match algorithm.
Each in turn makes a more flexible use of facets. Hierarchy and faceted navigation both require you to know the facets and their vocabulary, and to figure out how to navigate the systems. Search does not require any of that, but its results are far less precise. (Though we should note that with local search systems, you do need prior knowledge of the vocabulary of the space to be successful, because they don’t do the kinds of fuzzy matching that Google does.)
Hypertext links are also an expression of facets. They link from a topic to topics on related subjects, which are often facets of the current topic. So a wikipedia article on Ottawa has a link to an article on Canada, which is the location facet for Ottawa. Links, like search, don’t require prior knowledge from the user, and their results are perfectly precise, but they are not comprehensive — they only cover local connections from the present topic. Done well, they can be excellent within the context of one information search, but they don’t help much at all if you want to change to a different subject.
This is why navigation needs to be multi-modal. Each mode has its strengths and weaknesses.
Thanks, Mark! Yes, John shared a great metaphor for the non-hierarchical facets. But I’m glad that you painted the broader context and challenges too. From what I’m hearing, in essence, precise facets yield precise results, whereas ambiguous facets yield ambiguous results. Which makes perfect sense. In the case of Wikipedia, as you know, disambiguation mitigates the ambiguity. It might not be most elegant solution, but it certainly attempts to fill the gap.
Regarding hypertext links, there’s one exception to perfectly precise results — when the destination is actually a search query. For example, in the past, I’ve linked to YouTube search queries that take the form of www . youtube . com/results?search_query=albert+einstein. In this case, a precise result wasn’t as important as presenting the overall impression of results from which the reader can explore. Actually, I believe your SPFE discussion of “soft linking” follows a similar approach. But we can save that for a future discussion. 🙂
You’re welcome, Mark. Yes, authority, that’s it! That’s the word I was looking for. Not only does it address that sense of static indifference in hierarchical content, but it also pinpoints the source of that indifference. I agree, by comparison, the bottom-up nature of EPPO content should be more approachable too. You summed it up perfectly: “We just have to start a discussion… A post-hierarchical world says that the author does not get the final word on the organization or the significance of the content.” Unfortunately, in a litigious society, traditional product documentation still tends to rely on its final word.
Mark, can you comment on the different perspectives between this blog post and “Search is not enough” by the Nielsen Norman Group?
http://www.nngroup.com/articles/search-not-enough/?utm_source=Alertbox&utm_campaign=316a8bf232-Alertbox_email_09_08_2014&utm_medium=email&utm_term=0_7f29a2b335-316a8bf232-24076865
Thanks for the comment, Patty. You ask an excellent question. The short answer is, search is most definitely not enough. This is why I continually stress the importance of linking in a bottom-up information architecture. Each page should be a hub of its local subject space.
The long answer deserves a post in its own right. Stay tuned.
Pingback: Tech Writer This Week for October 2, 2014 | TechWhirl
Pingback: Search is Not Enough: Why We Need Multimodal Navigation | Every Page is Page One