Content producers often find collaboration a hard slog. This is because they do it wrong.
There are two fundamental ways you can collaborate. The first way is to enable everyone to contribute to the project by giving everybody access to everything and a means of communicating freely with everyone else on the team. I’ll call this the total integration model. When content folk say that they need to collaborate, this is almost always what they are thinking of.
The second way to collaborate is to break the project up into pieces and allow people to work independently on their own piece. To make this work, you have to set some constraints so that when each person finishes their piece, the pieces will all fit together. As long as each project team meets its constraints, the pieces of the overall project will mesh up with each other and the whole will work. Each piece of the project can be developed independently without people working on one piece needing to know what people in the other pieces are doing and without needing to look at the things they are working on. I’ll call this the constraint model.
Every significant commercial and industrial development project outside of the content space uses the constraint model. Most content development projects use the total integration model. Content creators are out of step with how the rest of the world collaborates, and it is costing them dearly.
What is wrong with the total integration model? Quite simply, the larger and more complex the project becomes, the harder it is for everyone to keep up to date on everything that affects their work. At a certain point, you end up spending all you effort simply trying to keep in touch with what is going on, with no time to do any new work. At this point, you either stop doing new work altogether, or you stop following everything that is going on in the project. When people stop following everything that is going on, you start to get duplication of effort, you start to get pieces that don’t fit, you start to get people going off by themselves on sub-projects that don’t contribute to the whole. In the end, you get a product that does not hang together, that does not really make sense, that does not work well or reliably or consistently.
The problem with total integration is that you drown in information. Total integration works by making all the information available to everyone. Constraint-based collaboration works by reducing to the the absolute minimum the amount of information that people need to know in order to collaborate. Rather than asking, “How can I give everyone access to everything?” the constraint model asks, “How can I restrict to a minimum what each person needs to know?” By reducing to a minimum the amount of information each person needs to do their work, you make their work simpler, and you reduce the overhead of collaboration so that there is more time to do new work. The secret to productivity is not more information; it’s less.
To be clear, for small teams working on small projects, the total integration model works. In fact, it is sometimes the only model that works for small-scale collaboration, because, up until a certain point in size and complexity, segmenting a project and defining constraints between the segments can be more overhead than simply having everyone sit round the same table and work together. Big projects that use the constraint model for the overall project often assign small teams to work on each segment, and those small teams collaborate using the total integration model.
Software projects provide great examples of the constraint model at work. Every major software project is segmented into separate systems or modules. These modules communicate with each other through a defined interface. Traditionally this has consisted of an API (Application Programming Interface), a set of function calls that one module can make to communicate with another. The API is a constraint, an agreement that if you want a certain service performed, you do it by calling a defined function in a defined way. You don’t need to know anything about how the function is implemented, and the people implementing the API don’t have to know anything about how your code works. As long as the API conforms to its spec, and you call the API in the way the spec says to call it, the system as a whole will work. This means that you have to know a lot less in order to get your work done.
Thanks to the API definition, two teams can work completely independently of each other without the need to know anything about what the other is doing. With less information and activity to keep track off, each team can spend more time on productive work. This constraint-based approach to software collaboration is so important that many of the developments in programming language theory and design, and in systems theory and design, can be thought of as attempts to make the segmentation of systems more complete and opaque so that the constraint-based collaboration model can work better.
Today, software development has turned more and more to another means of segmenting systems, replacing the calling of APIs with the exchange of messages. Independent modules communicate with each other by sending messages over a network. Web servers and web browsers, for instance, operate by exchanging messages using the HTTP protocol. Thanks to HTTP, web server developers and web browser developers don’t have to know anything about how each other’s products work, or which server or browser they are communicating with.
Message-based systems work really well for constraint-based collaboration, since the constraints that developers have to deal with are defined entirely by the message protocol. If you can read a correctly formatted message, and send a correctly formatted message, you can inter-operate with any other system that uses that message protocol. This allows the creation of mashups, hybrid applications built by interacting with multiple web services. If you thought reuse was cool, mashups are sub-zero.
Mashups would never happen if the people building all the systems involved had to meet and talk. The overhead would be too high. This kind of collaboration can only take place if the people involved do not have to communicate with each other. The secret to productivity is not more information; it’s less.
For many of these systems, the messages that they send each other are coded in XML. XML is a great tool for constraint-based collaboration. An XML schema could be described simply as a set of constraints. (In fact, there are a number of different schema languages for XML, including XML Schema, DTDs, RelaxNG, and Schematron. They differ from each other largely in the types of constraints they can express.) If you agree on an XML schema as a message format between parts of a system, you can work on the parts independently with the only constraint being that you read/write messages in the agreed schema.
Since content development uses XML more and more, you would think it would be well equipped to adopt a constraint-based model of collaboration. But alas, that is usually not what happens. When content projects get too big and too complex for the total integration model to work, they don’t segment and move to the constraint model. Instead, they either struggle along trying to make the total integration model work, and paying the price in overhead, or they disintegrate, slowly or quickly, into separate non-collaborating teams feeding separately off the carcass of the old content repository.
Why is it that truly massive software development programs can be successfully managed using relatively simple (and inexpensive/free) version control systems, while much smaller content projects cannot be managed without a much more complex and expensive content management system? And why is it that despite this additional complexity and expense, so many people are frustrated with their content management systems and struggling to find the productivity they were promised? It is because content management systems work on the total integration model of collaboration. They do not simplify or reduce the information needed to do the job, they simply make all the information more readily available. People drown in information. The secret to productivity is not more information; it’s less.
You might think that if you have adopted topic-based writing, that you are therefore doing constraint-based collaboration. After all, each writer writes their own topics independently. But the key to successful collaboration isn’t how you create the pieces, it is how the pieces go together to make the whole. Think about how a jigsaw puzzle goes together. Each piece is independent and has an interface — a combination of key and lock shapes that interlock to connect it to other pieces. A typical topic based writing system is designed to make sure that the pieces will fit together like jigsaw pieces.
But when you put a jigsaw together, you can’t just look at the shape of the connectors, you must also look at the bit of the picture on the face of the puzzle piece. In other words, the interfaces of the puzzle pieces are not opaque. The constraints imposed by the shape of the pieces does not succeed in hiding the information the piece contains. You have to look at the piece itself, not just the interface. The result is that when a team sets out to put a jigsaw puzzle together, they have to spread all the pieces out where everyone can see them, and then pass pieces back and forth from person to person as they gradually assemble the puzzle. In other words, doing a jigsaw is classic total integration collaboration.
This is generally what happens when teams set out to assemble information products from a collection of topics. Whether it be for reuse, for linking, or for assembling a document or a help system, everyone on the team needs to be able to see all the pieces, and you have to look at the pieces before you assemble them. In the case of many topic-based systems, where topics themselves are built up from other topics, this need to view all the pieces, and to track all the pieces, is not limited to the end of the project. It is something people have to do every day. Most topic-based writing projects work on the total integration model of collaboration, which is why they have such high content management overhead, and rely so heavily on big content management systems.
Is there an alternative? Can large content projects be run on the constraint model of collaboration? Certainly. Of that, anon.