Sometimes microblogging questions require macroblogging answers. Here’s the conversation:
@arh: Did @hixie really call XML a “disaster”? Is #techcomm aware of this? http://html5doctor.com/interview-with-ian-hickson-html-editor/
@mbakeranalecta: @arh XML is a disaster. Bad implementation of an essential concept. So is QWERTY. So it goes.
@arh: @mbakeranalecta Generally agreed. What concerns me are the #techcomm folks who think XHTML is “the future”.
@mbakeranalecta: @arh If people don’t see the value in making content mutable and addressable, then presentation formats are all they care about.
@arh: @mbakeranalecta Agreed that content must be mutable and addressable. But many in #techcomm dismiss the importance of presentation.
@mbakeranalecta: @arh Content must be mutable and addressable to improve presentation beyond what can be done by hand. Need both for progress in #techcomm.
@techwriterkai: @mbakeranalecta @arh Okay, I’ll take the bait: What does “mutable, addressable content” mean?
The answer to that won’t fit in a tweet, so here it is:
The sole purpose of all structured content technologies, XML, SGML, and all the rest, is to make content mutable and addressable. They serve no other purpose. Everything they enable is summed up in these two concepts: mutability and addressability.
To say that content is mutable is to say that it can be changed by an algorithm. (All content, of course, can be changed by a human author. We add structure so that we can hand the task of changing it off to an algorithm.)
To say that content is addressable is to say that you can direct an algorithm to a particular piece of content, or element within the content, so that it can change it. Addressability is absolutely necessary for mutability. If you cannot select and identify content to be changed using an algorithm, you can’t perform the change using an algorithm.
Consider the eternal XML example, a recipe. Here it is as HTML markup. I’m only including the ingredients list, which is enough to illustrate the point:
<html>
<head>
<title>Fried Eggs and Ham</title>
</head>
<body>
<h1>Fried Eggs and Ham</h1>
<table>
<tr>
<td>Eggs</td>
<td style="align:right">12</td>
</tr>
<tr>
<td>Butter</td>
<td style="align:right">2 oz</td>
</tr>
<tr>
<td>Ham</td>
<td style="align:right">1 lb</td>
</tr>
<tr>
<td>Salt</td>
<td style="align:right">1 tsp</td>
</tr>
</table>
</body>
</html>
This content is addressable. For instance, the address of the ingredient “Eggs” is the first cell of the first row of the table, which can be expressed using an XPath expression, like this:
/html/body/table[1]/tr[1]/td[1]
Evaluate that expression against the document and the result will be:
Eggs
The ability to address cells in a table can allow us to change their content. We could change how the content is formatted to make the first cell of each row bold, either like this, with XSLT:
<xsl:template match="td[1]"> <b><xsl:apply-templates/></b> </xsl:template>
Or like this, in CSS:
td:first-child
{
font-weight:bold;
}
Unfortunately, though, this algorithm will make every first cell of every table row bold. If all we want it to make the name of an ingredient in the list of ingredients bold, we don’t have a way to do the with an algorithm, because the markup does not give us a way to address ingredient names specifically. The ingredient name “Eggs” may have an address, but the address is too generic to be very useful. (There might be other ways for an algorithm to find ingredients, such as by natural language processing, but for the present purpose we are looking at how XML specifically enables algorithms by providing addressability, and this HTML sample does not provide enough addressability for more than basic indiscriminate formatting algorithms.)
It’s not just the format we can change once content is addressable. We can change the text itself. The following XSLT places the cell content in braces and forces it to lower-case:
<xsl:template match="td[1]"> <xsl:value-of select="concat( '[', lower-case(.), ']'"/> </xsl:template>
It that script were applied to the content, the word “Eggs” would be rendered:
[eggs]
Once you can address content, you can change it any way you want.
A little more addressability
Here is the same recipe marked up using DITA task markup (based on the sample at http://www.ibm.com/developerworks/xml/tutorials/x-ditaspecial/section2.html).
<!DOCTYPE task PUBLIC "-//IBM//DTD DITA Task//EN" "task.dtd">
<task id="r2">
<title>Fried Eggs and Ham</title>
<shortdesc>A simple meal to make any day.</shortdesc>
<taskbody>
<prereq>
<ul>
<li><p>12 eggs</p></li>
<li><p>2oz Butter</p></li>
<li><p>1lb Ham</p></li>
<li><p>1tsp Salt</p></li>
</ul>
</prereq>
<steps>
...
</steps>
</taskbody>
</task>
This example is slightly different from the HTML example, because here the author has chosen to use a list rather than a table for the ingredients list. This is an example of the presentation of the content being mutable by the author. An author could reformat the list as a table, but we could not create an algorithm to reliably change the display from a list to a table based on the current markup. The content does not have sufficient addressability to support that.
If you have to put rules in your style guide to ensure that all authors use a table to format the ingredient list of the recipes they write, and periodically have editors check the content to make sure that the writers have complied, it is because your markup does not give you sufficient addressability to allow an algorithm to format all ingredient lists as tables.
But does this example give us any more addressability or mutability than the HTML example? It gives us a little bit more. We know now that we are looking at a task. We could create an algorithms that built a publication which separated all the task topics from all the concept topics (supposing we thought that was a good idea). That is something we would have had to do by hand with the HTML markup. But because this markup does not specify what type of task it is, if your collection includes both recipe tasks and car repair tasks, you can’t write an algorithm to publish a cook book because it will include all the car repair tasks as well as the recipes. There is not enough addressability in the markup to tell fried eggs and ham from changing a flat tire.
The fact that the task is assigned an ID also makes it more addressable. From any other topic, the author can now address this topic by its ID, which can be used for linking or aggregating topics into a publication. But all the ID does is provide a unique identifier so that you can address the task unambiguously. It doesn’t tell you any more about what kind of task it is, so it really helps humans more than algorithms. The algorithm can assemble the topics that the human has listed for inclusion by their IDs, but the human still has to do the selection and aggregation by hand.
Below the level of the <task> element, though, we don’t get much more mutability or addressability than with the HTML example.
- The markup doesn’t say that the <ul> element contains a list of ingredients, so we can’t assign an algorithm to find all the recipes that use the ingredient “eggs”, for instance.
- We can’t assign an algorithm to create a shopping list from the four recipes I have chosen to make.
- The markup does not specify that the quantities of ingredients are quantities, so we can’t assign an algorithm to translate the quantities to metric measurements.
These are all ways in which we might want to change the content of a recipe, but the markup does not provide enough addressability to assign algorithms to do those changes. If they are going to be done, they will have to be done by hand.
As Scott Abel complains, all this kind of markup lets you do is to publish individual recipes. It does not let you do any of the useful manipulations described above, all of which could make the content more useful to the reader. Both the HTML markup and the generic DITA markup support some degree of addressability and mutability, but that addressability and mutability is pretty much limited to the addressability and mutability required to support a publishing process.
In fact, these generic markup systems don’t even support full mutability for publishing purposes. As pointed out above, you can’t assign an algorithm to change the list form of the ingredient list into a table form. If you want to do it, you have to do it by hand.
While XML is often sold on the idea that it lets you separate content from formatting, many systems are designed only for a limited range of publishing options in a limited number of media. As people are finding out when they try to publish to mobile, and to make changes to the display or inclusiveness of their content for delivery to mobile devices, their supposedly structured markup does not actually provide the degree of addressability necessary to assign the changes to an algorithm. This makes moving content to mobile a painful, tedious human activity.
We can greatly increase the mutability and addressability of content, and therefore the usefulness, if we give it some more specific markup, like this:
<recipe> <title>Fried Eggs and Ham</title> <ingredients> <ingredient> <name>Eggs</name> <quantity>12</quantity> </ingredient> <ingredient> <name>Butter</name> <quantity>2 oz</quantity> </ingredient> <ingredient> <name>Ham</name> <quantity>1 lb</quantity> </ingredient> <ingredient> <name>Salt</name> <quantity>1 tsp</quantity> </ingredient> </ingredients> ... </recipe>
With this markup we can now do all thing things listed above. For example, we can find all the recipes that use ham with an XPath like this:
/recipe[ingredients/ingredient/name='Ham']
And since the markup does not specify if the ingredient list is meant to be printed as a table or as a list, you can assign the chore of publishing them all consistently to a algorithm, which can work reliably now because the algorithm can tell what is a ingredient list, what is an ingredient, and what is a quantity.
Here’s the XSLT that would transform the recipe markup into HTML for publishing, using a table to display the list of ingredients:
<xsl:template match="ingredients"> <table> <tr> <th>Ingredient</th> <th>Quantity</th> </tr> <xsl:apply-templates/> </table> </xsl:template> <xsl:template match="ingredient"> <tr> <xsl:apply-templates/> </tr> </xsl:template> <xsl:template match="ingredient/name"> <td> <xsl:apply-templates/> </td> </xsl:template> <xsl:template match="ingredient/quantity"> <td> <xsl:apply-templates/> </td> </xsl:template>
It is now impossible for the author to get the formatting of the ingredient list wrong, so you can take that section out of the style guide, confident that the algorithm will always get the formatting correct. A slimmer style guide means a better chance that authors will apply the rest of its requirements consistently.
Although the markup does not separate the quantity value from its unit, the markup is actually sufficient to enable an algorithm to translate quantities to metric. It will require only elementary text analysis for the algorithm to separate the number from the unit. You should not ask humans to parse strings that algorithms can reliably parse for themselves.
There are all sorts of other things that you can do when you have this level of addressability in your content. You can automatically suggest substitutions from a table of substitutable ingredients. You can automatically create links to descriptions of ingredients. The more addressable the content becomes, the more mutable it becomes, and the more you can do with less effort.
Mutability and Exchange
There is something else that is really important about addressability and mutability, though. People often worry about exchanging information. They want to make provision for sharing their content with partners or ensuring that it will work with public systems. To accomplish this, they will often pick some public standard and create and store all their content in that standard.
There are three problems with this:
- Most public standards do not provide as much addressability and mutability as you might need in order to feed your other business processes.
- Most public standards are verbose and heavy and are a pain to write to.
- There is often more than one public standard that you might want to support.
The best way to support public standards, particularly if you may have to support more than one, is not to write in the public standard, but to write and store your content with the maximum amount of addressability and mutability, and then use an algorithm to transform it into whatever public standard format you need to deliver your content in. And because the public standard format is created by an algorithm, you will be able to achieve a much higher degree of consistency and conformance to that standard than if all your authors wrote in it directly. Mutability of the markup itself is one of the most important benefits that structured formats provide.
There is yet another benefit to using more addressable and mutable markup. Publishing-oriented standards like HTML and generic DITA tasks, require the author to make publishing-oriented decisions (like whether to format the ingredients list as a list or a table), which also means they need to understand publishing-oriented concepts, and be versed in your corporate publishing standards.
If forced to think about publishing and formatting issues as they write, most authors want to work in a WYSIWYG environment that lets them see if they are producing the proper publishing artifacts. If absolved of the need to think about publishing or formatting issues, however, authors can focus on content alone. This makes your full-time authors more productive, and makes it easier to engage experts as part-time authors and get good structured content from them without having to teach them the entire corporate style guide.
That, then, is what it means to make content mutable and addressable. If you want to automate any part of your authoring and publishing process, this is what you need to do to make it possible to apply algorithms to content. And the more mutable and addressable you make your content, the more automation you will be able to apply to it.
The Addressability – semantic, rather than merely structural – is essentially the embedding of an object model in the documentation content. So with your cookbook, we have Recipe, containing multiple Ingredients (usage, with quantities), of Products (type, units, labels such as ‘organic’), provided by Suppliers, etc, etc. Products and maybe Suppliers might be represented by separate topics. This then allows me to produce range of specific outputs – a Vegan Barbeque Cookbook, or whatever. It also, of course, allows me to query my content – my model, really.
The key to this is, as you say, the domain-specific markup – what would be a DITA domain – here for recipes and foodstuffs. I’m trying to move in this direction with our System Definition, and other, documentation.
Roy, yes, it is very reasonable to see this as embedding the object model inside the content, though I would make the point that the first consideration for the content model is that is should be designed first and foremost to get good data entry for authors. Asking authors to write into a model that is optimized for the computation of outputs is not likely to get you the most efficient or accurate authoring, so I would look at the content structures as being designed to capture from authors the data needed to create an object model for processing, not necessarily being that object model.
This, of course, is where the the mutability of content comes into play. So many systems are built as if markup was not mutable, and therefore require authors to work in a format that is designed as the input to a publishing engine, which, in turn, requires the author to know far more than they should about how the publishing engine works. DITA is particularly guilty of this, as is Docbook.
It should be the first principle of markup design that authors should not have to know anything about how the publishing system works. The authoring format should be designed for authoring and for authoring alone, and should be transformed behind the scenes into a format suitable for publishing.
Failure to do this often results either in the authoring fraternity being artificially reduced by the need to understand the publishing system (a problem also in FrameMaker, of course) or in the markup being reduced to such simplicity and generality that it can be created in Word, or something like Word, without any mutable or addressable structure being created.
I’m tickled to have motivated this blog post (I’m @arh on Twitter).
Building a bit on Roy, the previous commenter —
The DITA cognoscenti would likely propose that you overlooked the possibility of defining a DITA specialization to represent the “recipe” information model. Such a specialization would provide the semantic metadata necessary to support the addressability and mutability you describe, and provide the added benefit of leveraging DITA-aware authoring and publishing tools and content-management systems.
Do you have any reaction to the idea that the DITA specialization mechanism was designed specifically to support this scenario? Or is that another blog post? 🙂
Alan, thanks for the comment.
I frequently make the point that there are two DITAs. The first is the generic concept/task/reference DITA that you get more or less out of the box, and which carries with it the (unintentional?) implication that concept, task, and reference topics should inherently be kept separate (with which I strongly disagree — /2013/01/08/confusing-analytic-and-synthetic-truths-in-defining-topic-types/). It does not create addressable mutable content (as illustrated above).
The second DITA is the DITA of specialization, which is essentially another XML schema language alongside XSD, DTDs, Schematron, and RelaxNG. Like any of these, it can be used to create schemas for creating mutable and addressable content. And like any of these, its use does not automatically ensure you get mutable addressable content. That is all in the specifics of your schema design.
If you want to create mutable and addressable content, you can choose any of these schema languages to define your markup language. Of the five, DITA is by far the most complex and cumbersome, without having any more expressive power than any of the others. From a standalone point of view, it would make no sense to choose DITA for this purpose.
The one property that DITA has that these others do not is the ability to support what Eliot Kimber calls “blind exchange”. That is, a schema defined as a specialization of a base DITA topic type can be published by a tool chain designed for the base topic type. Not published well, necessarily. Not even published in any kind of comprehensible or readable form, necessarily, but the transformation will at least run and produce an output.
Is this a good thing or a bad thing? Personally, I find it hard to imagine a real world use case for blind exchange. It seems to me that if I could have no assurance of how a foreign DITA topic was going to be formatted by my tools, and if I had no knowledge of its specific semantics, I would rather just have a well formatted PDF produced by the originating system. But I leave it to others to decide if they have a use case for it.
The big difference I think blind exchange makes in practical terms is that it makes DITA a weakly typed schema language, where the other four are all strongly typed languages. (Weak typing means that one type can be automatically transformed to another type without raising an error, and strong typing means that all type transformations must be specific. Weak typing makes it quicker to write small programs, but can allows errors to go undetected. Strong typing requires more code but is more robust.)
In the programming world there are good reasons for having both weakly typed and strongly typed languages, so again, people will have to decide for themselves which fits their use case.
But for me, I always want my markup languages to be strongly typed. The point of creating mutable and addressable content is to allow content to be processed by algorithms, without the need for human supervision or review. To achieve that, you want to minimize the possibility of uncaught errors, and for that you want strong data typing.
So yes, DITA specialization is an option for creating mutable addressable content, but it is not the option I would choose in most cases.
(Sorry for the delayed response).
The kind of specialization I’m talking about in DITA is a ‘domain’ not ‘structural’ specialization. You specialize the keyword element to create a ‘system domain’ vocabulary. I work with a financial message-processing system, so there is a domain element for Interface, Message Type, Processing Stage, Alert, etc, etc. This is easy enough for authors to use – it’s a kind of ‘semantic highlighting – and is fairly easily extensible. In, say, XHTML output, the markup becomes a span with @class=’msgtype’, or whatever.
Hi Roy. Thanks for the comment. They are welcome at any remove.
I do a similar thing is SPFE with “mention markup”, though I use the native features of XML Schema to implement it (I’m not quite sure why DITA has to reinvent so much functionality that standard XML tools already provide. Perhaps it is a matter of age, as it predates namespaces and XML Schema which provide ways to do everything that specialization and the subject schema do, except blind exchange.) “Semantic highlighting” is a great was to describe it, and it is definitely a component of mutable and addressable content. It is not enough, by itself, to enable all the addressability and mutability you might want, but it certainly a big step.
DITA allows the specialisation of profiling attributes. Profiling attributes can be added to elements in topics, as well as to topicrefs in maps.
It’s simple enough for writers like me to understand and I think it could be useful for some basic publishing requirements. (Publish only the topics/sections with the attribute receipe=”lamb”.)
I’d be interested on your thoughts on this and how it relates to addressability.
On blind exchange, I’d never seen a custom specialised topic. It would be interesting to see a real example of a publication that combines the OASIS specialisations with some custom specialised topics.
Thanks for the comment John.
Yes, you can use attributes to provide addressability. However, I generally prefer to base addressability primarily on elements. One of the things you need to make addressability useful is consistency and reliability, so I look for ways to make it easiest to guide the writer and to provide feedback and validation of the content that needs to be reliably addressed. Elements offer more guidance and validation opportunities than attributes.
It very much comes down to the distinction between strong and weak topic typing, as I described it above in my reply to Alan. Attributes allow you to add addressability to a weakly typed or highly general schema (as in HTML microformats), but with less reliability. Custom authoring schemas specific to a particular kind of content offer strong typing and greater validation, as well as hiding content management and publishing issues from authors (something I think is an important next step of structured content: /2013/01/28/we-must-remove-publishing-and-content-management-concerns-from-authoring-systems/)