Tag: structured writing

Time to move to multi-sourcing

Single sourcing has been the watchword of technical communication for the last several decades. We have never fully made it work. A pair of seminal posts by prominent members of the community give me cause to hope that we may be ready to move past it.

Single sourcing is about the relationship between sources and outputs of content. Traditional publishing methods had a one to one relationship between sources and outputs, where each output was generated from a single source.

The inefficiencies of this model are evident, particularly where different outputs are just different formats of the same document, such as PDF and HTML. Single sourcing attempts to remove this inefficiency by combining content into a single source:



As Sarah O’Keefe comments in  Single-sourcing is dead. Long live shared pipes!, the first of the two seminal post I want to discuss:

[W]e have been trying for a world in which all authors work in a single environment (such as DITA XML, the Darwin Information Typing Architecture), and we pushed their content through the same workflow to generate output. We have oceans of posts on “how to get more people into XML” or “how to work with part-time contributors.” The premise is that we have to somehow shift everyone into the One True Workflow.

But there are two problems with this model. The first is that the source in this model is significantly more complicated than the individual sources in the original model. The added complexity comes from the need to support all of the potential outputs from a single source format and all of the management operations you want to perform on that content. But that complexity is imposed on everyone who writes any of the sources involved. Where people used to be able to write in Word or a simple visual HTML editor, they now have to write in Flare or DocBook or DITA or a complex CMS interface.

You can’t get everyone to agree that taking on this additional complexity is worth their while, and in practice it often slows down getting particular pieces of work done (even if it improves efficiency overall). So what happens is that some people drop out of the system, or refuse to sign on for it in the first place, or don’t use it for everything they do, leaving you with something that looks like this:

Life remains just as complex for the folks who continue to use the system, but the organization realizes fewer benefits from it because not everyone is using it.

Modern content demands have only made this situation worse. We now look to do more than simply issue content in multiple formats. We want to select and combine content to create different outputs for different audiences. We want to enhance content with metadata for consumptions by intelligent downstream systems. We want to richly link and organize content both statically and dynamically. We want to control terminology, manage quality, and steamline localization. All of this leads to greater complexity in your source format in order to support all of these things:

This added complexity is only going to result in more defections and dealing with the complexity, not to mention paying for the complex system, is only going to compromise your hoped-for ROI.

Is there another way?

O’Keefe suggests that the answer may lie in what she calls “shared pipes”. That is, a system in which content flows from many sources through a shared publication system and out to many different outputs.

This is a diagram I have been drawing for many years, and I love Sarah’s shared pipes analogy to describe it. It comes down to this:

There are many sources of content and many outputs, but the source material goes through a shared process in the middle that handles multiple outputs, selecting and composing for multiple audiences, adding rich metadata, rich static and dynamic linking, and all the rest.

But wait, don’t all those separate sources constitute that great boogeyman of the content industry, silos? Yes, they absolutely do, and that brings me to the second seminal post, Don’t Dismantle Data Silos, Build Bridges by Alan Porter. Porter begins by noting the reluctance of people to throw out their current way of doing things in favor of the one great single sourcing system:

Let’s face it: no one is going to throw out their incumbent systems just because we say they should. Especially not if those systems are still doing the job they were purchased to do. We have all worked with systems that are “good enough” to fulfill a specific set of tasks.

Removing and replacing existing systems isn’t quick or cheap, but the biggest hurdle isn’t budget, or technology (although that’s what’s often cited) — it’s human.

The human element is indeed crucial. Technically, the single sourcing model might work very well if it didn’t have to be used by humans. But, as Porter notes, humans build systems to suite their own work and they are not willing to give them up to make someone else’s work easier.

Nor should they, since the quality of a person’s work very much depends on the suitability of their tools to the task at hand. You don’t do your best brain surgery with tools designed for trimming hedges. The current systems that people are using may not be the best possible system they could be using, but at least those systems are specific to the work they are doing. They are comfortable. Perhaps we could design them a system that is even more comfortable, but if so it will be a system more specific to their task, not a single gigantic complex system designed to be a single source of everything.

So, a multi-source system lets us keep each source system focussed on the needs of the individuals who contribute to it. We then collect information from all those systems and run them through a shared publication process to produce whatever outputs we need. As Porter says:

Each customer interfacing system can still stand alone and address the needs of a particular line of business, or be an enterprise single source of truth. Yet by passing data between them, or existing enterprise business systems, they can be the foundation of a fully connected continuous customer experience.

Of course, it is not quite so easy as that. Setting aside the technical issues of actually connecting the various systems together, we are still left with the issue of whether the content coming from each of these systems has enough structure and metadata attached to it for the shared pipes to actually perform all the operations we need them to perform.

As O’Keefe points out, you don’t actually need every source to contain the structures to perform every system function:

not all content needs the same level of quality review. Instead, we’ll have make a distinction between “quick and dirty but good enough” and “must be the best quality we can manage.”

Thus it might be perfectly fine for some of your content to be written in a simple format like MarkDown and pass through your shared formatting pipe without necessarily passing through every other function and process of your central system.

But what about the content that does need to pass through  those functions. Can we support this without making every individual silo format as complex as our single source format:

Clearly if the silo formats have to be as complex as the single source format, we have accomplished very little. Even if each of these super-complex formats is specific to its silo, it is not likely to be supported by the existing system, nor is it likely to be comfortable for the individual contributor.

How then do we keep the formats of the individual silos small while still supporting all of the functions of the central publishing system for all the types of content that need it?

The answer lies in structured writing, particularly in the style of structured writing that I call “subject domain.” In subject-domain structured writing, the markup you add to the text is specific to the subject matter. Here, for example, is a recipe in subject-domain a structured writing format:

recipe: Hard-Boiled Egg introduction: A hard-boiled {egg}(food) is simple and nutritious. ingredients:: ingredient, quantity, unit eggs, 12, each water, 2, qt preparation: 1. Place eggs in (pan){utensil} and cover with water. 2. {Bring water to a boil}(task). 3. Remove from heat and cover for 12 minutes. 4. Place eggs in cold water to stop cooking. 5. Peel and serve. prep-time: 15 minutes serves: 6 wine-match: champagne and orange juice beverage-match: orange juice nutrition: serving: 1 large (50 g)   calories: 78 total-fat: 5 g saturated-fat: 0.7 g polyunsaturated-fat: 0.7 g monounsaturated-fat: 2 g cholesterol: 186.5 mg sodium: 62 mg potassium: 63 mg total-carbohydrate: 0.6 g dietary-fiber: 0 g sugar: 0.6 g protein: 6 g read more

Structured Writing and Free Trade

In my last post, I promised I would reveal the unifying idea that I developed for my new book on Structured Writing. This is the post. So what does it have to do with free trade? Mostly it is that I see the same pattern in discussions of free trade that I do in many discussion of structured writing: a failure to focus on the big picture.

Free trade has been pretty much a given for the last several decades with nations and trading blocks negotiating ever freer trade. But of late the virtues of free trade have been called into questions by the Trump/Sanders wing of American politics, which has those of us in Canada, for whom America is our largest trading partner, a little anxious. read more

Designing for Feedback

We were discussing the biggest challenges in Tech Comm at the last STC Toronto brunch and we all seemed to agree that the difficulty getting feedback on the effectiveness of the content we create is the biggest challenge. The things that really matter in technical communication is whether users can achieve their goals after finding and reading the docs, and whether that makes a difference in whether they buy from you again or speak well of you to their friends and colleagues. These things are incredibly difficult to measure. You are just not there to see them happen, and it is very difficult to separate their effects from the effects of the work that designers, developers, sales people, trainers, field engineers, and support staff do. read more

Is There a Reproducible Method for Explanation?

In a recent LinkedIn discussion on “Most important competencies for technical writers“, I commented that the most important skill for technical writers was explanation, and that the ability to write and the ability to explain are not the same thing, and that the ability to explain is significantly less common that the ability to write well enough.

This scarcity of natural ability to explain well, I argued, is why we need to pay attention to tools and structure, because we need to find ways to help people who do not have a natural gift for explanation to explain things at least reasonably well. Tacitly, this argument suggests that there is a reproducible method for explaining things. read more

The Metadata is Bigger Than the Data

It is easy to think of metadata as a few information fields that you tack onto an article when you submit it to a CMS. In this vision, metadata is something fairly small, a triviality in comparison to the content itself.

The reality is just the opposite; for any piece of content or data, the metadata is bigger than the data. Metadata is data that explains data. And since data needs to be explained to be useful, metadata also needs metadata to explain it. This means that behind every piece of data there is a metadata cascade that eventually leads back to unstructured stories. Therefore the metadata for any given piece of data or content is all of the data that explains the entirety of that data/content. That’s a lot. read more

What kind of “easy” authoring are you looking for?

I was reading JoAnn Hackos article on easy DITA authoring solutions and it got me thinking about what the word “easy” means in regard to DITA or any similarly complex technology.

Can an editing interface make DITA easy? Some DITA consultants that I know complain bitterly about tools that make that claim. DITA may be many things, they say, but easy is not one of them. To present it as being easy is only going to set people up for disappointment and failure.

If they are right, can any DITA editor legitimately claim to be “easy”? read more

Structured Writing is Essential for Developer Docs

Tom Johnson wrote a post recently in which he questioned the value of structured writing for developer documentation. Needless to say, I disagree. But Tom and I are not really at odds here. Rather, he means something different by “structured writing” than I do.

Structured writing is about content quality, not publishing

What I mean by “structured writing,” and what structured writing has traditionally meant, is essential for developer docs. But it is something different from what “structured writing” has come to mean in tech comm of late, as we can see from Tom’s argument: read more

Designing topic types

A number of readers have asked me to write about how to design a topic type. Although it can sound complicated, especially if your bring XML schema definitions into the mix, designing a topic type is actually pretty simple.

Before we begin, though, set aside all the issues around XML. XML has nothing to do with topic type design. It may have a lot to do with how you implement your topic types (or nothing to do with it), but it has nothing to do with how you design them.

Needed information in a specific form

A topic type is essentially two things: read more

XML is Not the Answer

XML is not the answer. Structured writing may be the answer. XML is one way to implement structured writing.

More and more these days, I am hearing technical publication managers (and not a few consultants) talk about the need to “move to XML”. This may be a shorthand way for them to describe a very sensible plan to implement structured writing, but if so, I wish they would say “move to structured writing”. Structured writing is a potential answer to many content challenges. XML, by itself, is not.

There is a very real danger in all this XML talk that people can get the impression that if they move their content to XML — any XML — then they will instantly have the ability to reuse content across the enterprise and to seamlessly and effortlessly deliver content to any and every channel in a form that perfectly matches the optimal information design for that channel. It just isn’t so. read more

Changing the “what” in WYSIWYG

The “what” in “what you see is what you get” is not always “formatting”.

Authors need WYSIWYG (What You See is What You Get) displays to work effectively. This is basic ergonomics. If you can’t see the thing you are supposed to be creating, you will have a hard time creating it well.

But while authors always need WYSIWYG displays, the “what” that they need to see is not the same thing in every case. There can be several different values for the “what” in WYSIWYG.

The term WYSIWYG was coined to describe the displays of word processing and desktop publishing programs. In this case, the “what” was the formatting of the printed document. (At the time we were talking strictly about printing on paper. The “what” you saw on screen was the “what” that was going to appear on paper — more or less.)

This was a big step forward, but the step forward was not enabling people to do typesetting with computers. Doing typesetting with computers had been possible for a long time using a language called TeX, which is still used in some places today. But TeX, has never been widely used because while you can specify formatting with great precision in TeX, you can’t see the formatting that you are specifying as you work. You can only see it afterward when you run the file through the TeX program to create formatted output.

Make a mistake in your TeX markup, and your output may be garbled in ways that make it hard to find your error, and finding and fixing errors can be fiddly and tedious. Desktop publishing was a big step forward because it allowed people to see the formatting they were creating as they created it.

But formatting is not always the “what” in WYSIWYG. In Learning Python, Mark Lutz describes the syntax of the of the Python programming language as WYSIWYG.

Python is a WYSIWYG language — what you see is what you get — because the way code looks is the way it runs, regardless of who coded it.

Lutz, Mark (2013-06-12). Learning Python (p. 326). O’Reilly Media. Kindle Edition.

What he means in this case is that with Python, unlike most other programming languages, the indentation of the source code always shows the logical structure of the program. If formatted poorly, the indentation of code in a C or Java program can be completely misleading as to the actual structure of the code. Lutz provides this example:

if (x)
    if (y) 
        statement1; 
else 
    statement2;

Here the formatting of the code makes it appear as if the else statement belong to the first if statement, but it doesn’t. It actually belongs to the second if statement. The structure you see is not the structure you get. In Python, such confusion cannot occur. Lutz’ Python example looks like this:

if x: 
    if y: 
        statement1 
else: 
    statement2

In this case, the else does belong to the first if, as it appears. If it were indented under the second if, like this:

if x: 
    if y: 
        statement1 
    else: 
        statement2

then the else would belong to the second if statement

In Python, the indentation of the source code determines the structure of the program, and thus the logical structure you see in the source is the actual structure you get in the program. Thus Lutz chooses the term WYSIWYG to describe the language. But in this usage of WYSIWYG, the “what” is not formatting, but structure.

In content engineering there are multiple possible “whats” you might want from a WYSIWYG display. Desktop publishing is based on the premise that the author is also the designer, the typesetter, and the prepress person, so the “what” is formatting. We can be more explicit about this by substituting the word “formatting” for the word “what” in the acronym: FYSIFYG (the Formatting You See Is the Formatting You Get). The desktop publishing approach requires a FYSIFYG display.

Before the days of desktop publishing, when authors were only required to worry about the text, and did not have to wear a dozen other hats, they got by with pens or with typewriters. Both these tools had a TYSITYG display: the Text You See Is the Text You Get.

Today, however, the increasing demand for malleable and repurposable content means neither TYSITYG not FYSIFYG will suffice. We need more than just the text, and something other than text plus formatting. We need structure. What we need to create structured content, therefore, is a SYSISUG display: the Structure You See Is the Structure You Get. Creating such a display, alas, has not been easy.

It might seem like a raw XML file is a SYSISUG display, because all the structure is there in the XML tags. Many XML editors that let you switch to a view of the raw XML will call this “the structure view”. The problem is, while the structure is there, it is not in the least easy to view. There is way too much syntactic noise in an XML file for the reader to easily discern the structure it expresses, as this chunk of the DocBook source for my book shows:

<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0" xml:id="chapter.linkrichly"> <title>EPPO Topics Link Richly</title> <indexterm><primary>Weinberger, David</primary><secondary>on linking</secondary> </indexterm> <blockquote> <attribution>David Weinberger, <citetitle pubwork="book">Too big to know</citetitle> <biblioref linkend="TooBigToKnow"/></attribution> <para><indexterm class="startofrange" xml:id="idx.linking-1"><primary>linking</primary></indexterm>Links are the visible manifestation of the author giving up any claim to completeness or even sufficiency; links invite the reader to browse the network in which the work is enmeshed, an acknowledgement that thinking is something that we do together.</para> </blockquote> read more