Content producers often find collaboration a hard slog. This is because they do it wrong.
Total integration collaboration connects everyone to everyone and everything in the project.
There are two fundamental ways you can collaborate. The first way is to enable everyone to contribute to the project by giving everybody access to everything and a means of communicating freely with everyone else on the team. I’ll call this the total integration model. When content folk say that they need to collaborate, this is almost always what they are thinking of.
The second way to collaborate is to break the project up into pieces and allow people to work independently on their own piece. To make this work, you have to set some constraints so that when each person finishes their piece, the pieces will all fit together. As long as each project team meets its constraints, the pieces of the overall project will mesh up with each other and the whole will work. Each piece of the project can be developed independently without people working on one piece needing to know what people in the other pieces are doing and without needing to look at the things they are working on. I’ll call this the constraint model.
In the constraint model, each person works individually to a set of defined constraints.
Every significant commercial and industrial development project outside of the content space uses the constraint model. Most content development projects use the total integration model. Content creators are out of step with how the rest of the world collaborates, and it is costing them dearly.
What is wrong with the total integration model? Quite simply, the larger and more complex the project becomes, the harder it is for everyone to keep up to date on everything that affects their work. At a certain point, you end up spending all you effort simply trying to keep in touch with what is going on, with no time to do any new work. At this point, you either stop doing new work altogether, or you stop following everything that is going on in the project. When people stop following everything that is going on, you start to get duplication of effort, you start to get pieces that don’t fit, you start to get people going off by themselves on sub-projects that don’t contribute to the whole. In the end, you get a product that does not hang together, that does not really make sense, that does not work well or reliably or consistently.
The problem with total integration is that you drown in information. Total integration works by making all the information available to everyone. Constraint-based collaboration works by reducing to the the absolute minimum the amount of information that people need to know in order to collaborate. Rather than asking, “How can I give everyone access to everything?” the constraint model asks, “How can I restrict to a minimum what each person needs to know?” By reducing to a minimum the amount of information each person needs to do their work, you make their work simpler, and you reduce the overhead of collaboration so that there is more time to do new work. The secret to productivity is not more information; it’s less.
Sometimes small teams using total integration work with other teams via constraint-based collaboration.
To be clear, for small teams working on small projects, the total integration model works. In fact, it is sometimes the only model that works for small-scale collaboration, because, up until a certain point in size and complexity, segmenting a project and defining constraints between the segments can be more overhead than simply having everyone sit round the same table and work together. Big projects that use the constraint model for the overall project often assign small teams to work on each segment, and those small teams collaborate using the total integration model.
Software projects provide great examples of the constraint model at work. Every major software project is segmented into separate systems or modules. These modules communicate with each other through a defined interface. Traditionally this has consisted of an API (Application Programming Interface), a set of function calls that one module can make to communicate with another. The API is a constraint, an agreement that if you want a certain service performed, you do it by calling a defined function in a defined way. You don’t need to know anything about how the function is implemented, and the people implementing the API don’t have to know anything about how your code works. As long as the API conforms to its spec, and you call the API in the way the spec says to call it, the system as a whole will work. This means that you have to know a lot less in order to get your work done.
Thanks to the API definition, two teams can work completely independently of each other without the need to know anything about what the other is doing. With less information and activity to keep track off, each team can spend more time on productive work. This constraint-based approach to software collaboration is so important that many of the developments in programming language theory and design, and in systems theory and design, can be thought of as attempts to make the segmentation of systems more complete and opaque so that the constraint-based collaboration model can work better.
Today, software development has turned more and more to another means of segmenting systems, replacing the calling of APIs with the exchange of messages. Independent modules communicate with each other by sending messages over a network. Web servers and web browsers, for instance, operate by exchanging messages using the HTTP protocol. Thanks to HTTP, web server developers and web browser developers don’t have to know anything about how each other’s products work, or which server or browser they are communicating with.
Mashups take advantage of well-defined protocols to build hybrid applications - a form of constraint-based collaboration.
Message-based systems work really well for constraint-based collaboration, since the constraints that developers have to deal with are defined entirely by the message protocol. If you can read a correctly formatted message, and send a correctly formatted message, you can inter-operate with any other system that uses that message protocol. This allows the creation of mashups, hybrid applications built by interacting with multiple web services. If you thought reuse was cool, mashups are sub-zero.
Mashups would never happen if the people building all the systems involved had to meet and talk. The overhead would be too high. This kind of collaboration can only take place if the people involved do not have to communicate with each other. The secret to productivity is not more information; it’s less.
For many of these systems, the messages that they send each other are coded in XML. XML is a great tool for constraint-based collaboration. An XML schema could be described simply as a set of constraints. (In fact, there are a number of different schema languages for XML, including XML Schema, DTDs, RelaxNG, and Schematron. They differ from each other largely in the types of constraints they can express.) If you agree on an XML schema as a message format between parts of a system, you can work on the parts independently with the only constraint being that you read/write messages in the agreed schema.
Since content development uses XML more and more, you would think it would be well equipped to adopt a constraint-based model of collaboration. But alas, that is usually not what happens. When content projects get too big and too complex for the total integration model to work, they don’t segment and move to the constraint model. Instead, they either struggle along trying to make the total integration model work, and paying the price in overhead, or they disintegrate, slowly or quickly, into separate non-collaborating teams feeding separately off the carcass of the old content repository.
Why is it that truly massive software development programs can be successfully managed using relatively simple (and inexpensive/free) version control systems, while much smaller content projects cannot be managed without a much more complex and expensive content management system? And why is it that despite this additional complexity and expense, so many people are frustrated with their content management systems and struggling to find the productivity they were promised? It is because content management systems work on the total integration model of collaboration. They do not simplify or reduce the information needed to do the job, they simply make all the information more readily available. People drown in information. The secret to productivity is not more information; it’s less.
You might think that if you have adopted topic-based writing, that you are therefore doing constraint-based collaboration. After all, each writer writes their own topics independently. But the key to successful collaboration isn’t how you create the pieces, it is how the pieces go together to make the whole. Think about how a jigsaw puzzle goes together. Each piece is independent and has an interface — a combination of key and lock shapes that interlock to connect it to other pieces. A typical topic based writing system is designed to make sure that the pieces will fit together like jigsaw pieces.
Assembling a jigsaw puzzle requires everybody to see everything and work together constantly.
But when you put a jigsaw together, you can’t just look at the shape of the connectors, you must also look at the bit of the picture on the face of the puzzle piece. In other words, the interfaces of the puzzle pieces are not opaque. The constraints imposed by the shape of the pieces does not succeed in hiding the information the piece contains. You have to look at the piece itself, not just the interface. The result is that when a team sets out to put a jigsaw puzzle together, they have to spread all the pieces out where everyone can see them, and then pass pieces back and forth from person to person as they gradually assemble the puzzle. In other words, doing a jigsaw is classic total integration collaboration.
This is generally what happens when teams set out to assemble information products from a collection of topics. Whether it be for reuse, for linking, or for assembling a document or a help system, everyone on the team needs to be able to see all the pieces, and you have to look at the pieces before you assemble them. In the case of many topic-based systems, where topics themselves are built up from other topics, this need to view all the pieces, and to track all the pieces, is not limited to the end of the project. It is something people have to do every day. Most topic-based writing projects work on the total integration model of collaboration, which is why they have such high content management overhead, and rely so heavily on big content management systems.
Is there an alternative? Can large content projects be run on the constraint model of collaboration? Certainly. Of that, anon.
Very interesting post, thank you, Mark!
One reason the constraint model goes often fails is that participants don’t know that constraints are based on roles or tasks. And each role or task comes with its own set of constraints – which until not too long ago formed the backbone of independent professions, such as editors, indexers, etc.
“The secret to productivity is not more information; it’s less.” That is also true for our content consumers and gave rise to minimalist documentation (so maybe it’s not so secret anymore…?).
I don’t think that total integration and the constraint model are the only two choices you have to organise content collaboration. I’ve had good results in environments which allowed total integration – as long as you heeded the constraints per role. I’m not sure whether this is what you had in mind when you talk of big constraint-model projects including smaller total integration teams.
I don’t think that the jigsaw metaphor works, for several reasons:
– Putting together a puzzle is an exercise in space, virtually regardless of time and sequence. Putting together topics in a map arranges a reader’s experience in time and sequence. Even though it’s still called a map, topics are not really arranged in space – which is why every page is page one…
– The number of plausible connectors is variable across time as you add or delete links between topics.
– Topic-based content can use “glue topics” which mediate between connectors and help readers navigate between topics.
– If you need an all-hands meeting to pass the topics back and forth to piece the puzzle together, you have a problem with your topic titles and/or topic tags: From titles and/or tags, I should be able to assign 90% of the topics to the person who knows how to organise them, because they belong to her “corner”.
Hi Kai,
Thanks for the comment. Certainly dividing tasks into roles is one kind of constraint-based collaboration. The classic separating-content-from-formatting theme found in many forms of structured writing is a perfect example of this. The structured schema is a constraint that separates the writing chore from the formatting chore. It means that the writer does not have to worry about what the content will look like when published, thus freeing them to work on other things.
But there is also the dividing of tasks into sub projects where you remove or reduce the need of people playing the same role to have to communicate with each other or look at each other’s work. Without getting too far into my planned follow-up post on how to do constraint based collaboration for content, the soft linking technique that I described in an earlier post is one way to do this.
I agree that doing a jigsaw is not a perfect allegory for creating a topic map. I meant it only as a general metaphor for this way of working. The assigning of certain areas to a particular individual, though, is something one sees in jigsaw puzzles as well. Total integration does not mean that everyone has to touch everything or agree on everything, it only means that anyone may need to see any particular thing, or to reach and agreement with one or more colleagues on one of more things. Some total integration projects can therefore work much more efficiently than others. On the other hand, there is no way to know if all the necessary communication and consultation actually has taken place. Defining constraints ensures that unnecessary communication is eliminated, but it also helps ensure that all necessary communication does take place.
Interesting read. I’m waiting for part two. 🙂
Hi Mark,
Thanks for sharing your thoughts on this, and for writing so lucidly about a complex issue. I’m working on a content (management) strategy project for a large public sector client at the moment, and I found the post particularly useful because it shines a light on the culture-change aspect of this work, which is so often the most challenging one.
Implementing a structured approach to content creation is, I think, an important lever for changing team members’ expectation from ‘I need to know/control everything’ to ‘Just tell me what I need to know’. I found your comparison of the two approaches to collaboration enlightening (even if they aren’t the only ones – a fact of which I’ve no doubt you are well aware). Back to my content models now, but I’ll certainly be exploring your posts in more depth.