Don’t just try to fit into development’s agile process. Create your own lean content development process.
Technical writers are increasingly finding themselves involved in the agile process of the development organization. The most common way this happens is that a writer gets assigned to the team for a sprint and is expected to produce documentation in the same sprint as the developers produce the feature.
This has one huge plus for writers: development cannot declare the sprint complete until the documentation is done. This means that it is in the interest of every developer to help ensure that the tech writer has what they need to get their work done. Given that getting information when needed is a major issue for many tech writers, this can be a very big win.
But there are plenty of downsides to this arrangement as well. Some of the more common complaints include:
- There are often not enough writers available to assign one writer to each sprint team full time, so the writer ends up being assigned to several teams. Attending multiple scrum meetings can then take up a big chunk of the day, and juggling the priorities of multiple teams becomes a significant headache. Also, having to switch gears between different features can inhibit the writer’s flow, greatly increasing the time it takes to write the content.
- It does not always take the same amount of writing time to document a feature as coding time to implement it. Some features that have major architectural impact but little user-visible behavior can take weeks to code while requiring little or no documentation. Small changes to user-visible parts of the system, on the other hand, may require major documentation effort while being implemented with very little code. The writer trying to work in concert with the sprint team may be scrambling one week and twiddling their thumbs the next.
- Documentation is at the mercy of development’s process and schedule, which is not being optimized for the creation of documentation, but of software. And, of course, not every organization implements agile well. Being at the mercy of a good process designed for something else is not ideal. Being at the mercy of a bad process designed to produce something else is very difficult.
- Tying all docs development to software development sprints may encourage product oriented rather than task oriented or user oriented documentation. Petko Mikhailov makes this point in a post on LinkedIn where he notes:
My experience has been that Agile methodology naturally leads to product-centered documentation and it is quite a struggle to maintain the user perspective and the needed content structure.
This is ironic because Agile has been designed to better cater user needs; it however doesn’t seem to recognize that documentation has its own needs that are separate from the product development programming needs.
I’m certain there are writers who are integrated into an agile team and who do feel that their needs are being considered. But while this may be true, Mikhailov’s point is also true: documentation’s needs are separate from development’s needs. Successful integrations may owe more to the flexibility and thoughtfulness of the people involved than to the inherent virtue of integrating documentation into development’s agile process.
This is why I always advise tech comm groups not to simply integrate writers into development’s agile process, but to develop their own lean/agile process and optimize the interface between their process and development’s process.
I see this more in lean terms than agile terms, because agile is designed specifically for software development, whereas lean is more general. Agile, however, can be seen as the application of lean principles to software development, so the two approaches are broadly compatible.
What lean tells us is to focus on value and on the value stream. Documentation is a distinct product with a distinct value set to deliver. It’s units are not necessarily the same as development’s units. Inserting tech comm into development’s process involves inserting tech comm into development’s value stream. The value that development produces is features, so naturally there will be a tendency for tech comm to end up documenting features, and it will be hard to find time to develop docs that are not tied to a specific feature, as Mikhailov notes.
Creating your own lean documentation process lets you focus on your own value stream, to deliver your own value, which is providing task assistance to users. Set up that value stream first, then start to ask what is the right interface to the development process to get the information you need to create value in your process.
Also, lean is a set of principles rather than a set of practices. Agile practices were developed for software development, and may or may not be appropriate, separately or as a whole, for other processes. Following lean principles allows you to develop practices that are optimal for content creation rather than software development.
One of the most important benefits of taking this approach is that it forces you to think in terms of making your entire documentation process lean, as opposed to simply adapting what you do now to fit into the agile mold dictated by development. Indeed, I would urge all tech comm groups to undertake a lean transformation of what they do, regardless of whether their development organization is practicing agile or is still using a waterfall methodology.
How to do a lean transformation of your docs department is a larger topic than one blog post can handle, but here are the slides for a presentation on lean content development that I gave at Lavacon 2013:
Mark, this is an interesting subject. Thanks for elaborating with depth. A couple of points caught my attention. On this slide 30 in the slideshare embed – http://www.screencast.com/t/eUQWQJxi2DN — you say to start later in the process rather than earlier. This is a tricky decision.
Many tech writers I’ve known frequently complain that they don’t get earlier access or involvement. But earlier access often means you document the feature as it evolves, which leads to waste.
On the other hand, getting access late in the game means that when you discover interface problems (poor word choices in the UI, for example), or massive amounts of workflow confusion, it’s too late to stop or change it, since the feature is already slated for a release and no engineer wants to change it.
How do you find a balance between the two predicaments? It’s like a damned-if-you-do, damned-if-you-don’t situation.
Another slide caught my attention — http://www.screencast.com/t/85Qt09rlh1v (slide 31). Do the simplest thing that works today. In my situation, we write and review content in Google docs formatted as Markdown and then convert the Markdown to HTML via Stackedit and publish to Drupal.
While this works for our current situation — 2 writers, approx. 5-10 pages of new content every 2 weeks, no need for PDF or translation — we think that in the future, we may suddenly be faced with translation scenarios, or some other customer may decide they need us to produce a PDF.
At that point, with none of our content in XML and no real multichannel publishing strategy established, won’t we be overwhelmed with the request? Shouldn’t we prepare now for this future possibility? Or is that kind of anticipated planning against unknown future requirements a waste, since those requirements may never come to light?
Tom
Thanks for the comment, Tom.
One of the things I like about Lean is that it emphasises principles rather than practices. Start as late as possible is a principle. In practice, “as late as possible” may be relatively early in some processes and much later in others.
Another principle of Lean is evenness, and in many ways it is the principle that governs all the others. Starting late is good because more information is available late, but if it causes a big lump of work late in the process, that is bad because it is uneven. You have to find the balance that produces evenness.
That said, there is no reason why the tech writer cannot discover the kinds of issues in the product that you are talking about, the ones that will cause usability problems and/or documentation problems, early in the process, but postpone writing about the feature until later. Examining the value chain may lead us to the conclusion that the tech writer’s role as first user does not have to be tied to their writing schedule. As first user, the writer generated design feedback, which is valuable early in the process. But writing about an immature design may be wasteful because the design will change. So split the two activities: use early; write late.
Tom and Mary Poppendieck also note that the principle of starting late has to be tied to the ability to execute quickly. The quicker you can execute, the later you can start. But tech pubs traditional processes make executing quickly very difficult. All the buzz around structured writing today seems to be about reuse, but while reuse can help you execute quickly, I would prefer to see a broader focus on rapid execution as a key reason to adopt structured writing.
As far as doing the simplest things that works is concerned, there is a trap that many documentation groups fall into when it comes to XML. They believe that creating content in XML will enable them to prepare for future possibilities. Alas, it won’t. By itself, XML is just a file format. It is not the XML that enables additional capabilities, but the specific semantic structure that you capture. What advanced capabilities you can develop is a function of what semantic structure you capture, regardless of file format.
Unless you know what capability you are going to want, you can’t specify the structure that will give you that capability. So going to XML without a specific capability in mind is probably not going to result in your having whatever capability you may decide you need in the future.
The only real future proof approach to structure is pure semantic structure. That is, a structure that does not relate to any system functionality, but only describes the semantics of the subject matter. The problem is that as long as such structure is not being used, it tends not to be created accurately, and not to be validated rigorously, meaning that when you come to use the content in a new process, it is not actually reliable enough to work in that process.
You can, of course, make a wager on a particular technology, hoping it will still be current and that the capability it provides will be the capability that you want when you figure out what you do want. But, depending on the cost of said technology and its implementation, that can be an expensive bet to lose.
Thank you, Mark, for the article. I agree that we should have our own way of managing the documentation tasks. In my previous stint, we would follow the “development -1” (development minus 1) sprint for preparing our first pass and documentation -2 for preparing our review-incorporated documents.
Alternatively, you can follow document-specific sprints that run in parallel to the development sprints. I agree with Mikhailov’s opinion that documentation’s needs are different from development’s needs.
I have experienced that preparing documentation in agile becomes difficult when there are feature-related changes in every build. Also, agile is good for companies that expect changes based on customers’ feedback. For documentation though, it becomes difficult to incorporated customer-specific or feature-related changes.
Thanks for the comment, Suyog.
If you adopt a lean content development strategy, you will need to figure out how to create the best information flow between development and docs. Working one or two sprints behind is certainly one way of doing that.
Whether that means working in sprints on the doc side or not is a different question. A kanban approach, for instance, might make more sense than working in sprints.
The point of developing your own lean process is that you can adopt the methodology that makes the most sense for producing docs, regardless of how development works.
We’ve come to a similar conclusion (here’s my slide deck from earlier in the year http://www.slideshare.net/ellispratt/leantalk-lightning2013 and there’s a whitepaper on Adobe’s TCS site). Agile tries to avoid the waste of rework, but in the world of documentation that can lead to greater wastes of Mura (unevenness) and Muri (overwork).
Rats – that’s the wrong slidedeck. Here are the correct links
http://blogs.adobe.com/techcomm/2013/06/towards-an-agile-authoring-methodology-learning-from-the-lean-methodology.html
https://www.madcapsoftware.com/demos/signup.aspx?id=1134255707524019108
Thanks for the comment, Ellis. Thanks for the links as well. Those are useful resources.
Indeed, I think the principle reason to separate doc’s lean process from development’s agile one is to avoid Mura and Muri. An agile sprint, even if planned perfectly, reflects development’s cadence, not documentation’s cadence, which is usually different. That difference introduces Mura and Muri into the writer’s life, which is wasteful.
As Donald Reinertsen says, there are no best practices. Any specific practice is only best in the specific situation in which it operates. Development’s best practice is unlikely to also be documentation’s best practice.
A related point here is that including docs in the dev sprint can introduce Mura and Muri into development’s process as well. If docs is overworked in a sprint (Muri) and the sprint can’t end until docs finishes, that will force dev to wait (Mura).
One of the features of my current project is a 90 minute scrum. I got a recliner chair. Home office. As an employee, this would be pure madness. As a consultant, it fascinates me.
Thanks for the comment, Tina.
Is that a daily meeting? There is a reason that these meetings are often held standing up. (http://en.wikipedia.org/wiki/Stand-up_meeting) One place I worked even frowned upon people leaning on the wall during them.
Hi Mark,
Some really good points in this post. I especially like the lean principles about focusing on value.
I have a bit of a different take on development’s versus documentation’s needs, however. I see documentation as part of the product, rather than a separate product. If you are incorporating practices in your release cycle such as iterative usability testing, this will improve the overall product, and hopefully reduce the need for documentation.
As far as the tendency to document *features* versus documenting tasks in an agile environment, I think this will depend on the maturity of your documentation set and team. Our team works to target our documentation on users’ goals and the tasks they perform to complete those goals. So if Dev is working on implementing a feature, we look at how that feature fits into overall workflow, help with usability of the feature to that end, then document it in the context of how they are using that feature *to do their job.*
Finally, using a whole team approach will help teams when they have either dev-heavy or doc-heavy user stories. We have had writers help with testing, developers write first drafts of documentation, or testers set up VMs for writers, all so the various functional areas stay together in their sprints. Something to consider.
Thanks again for a thorough post that makes me want to go investigate lean some more.
Thanks for the comment, Alyssa. There are certainly upsides as well as downsides to working in development’s sprints. My point is less about avoiding this on principle and more about having your own lean process, where you think about, and apply lean principles to, everything you do in documentation. In some cases, the greatest evenness and flow may indeed by achieved by working in the same sprints with development, though that should be arrived at as a conclusion from the leaning of documentation rather than taken as a necessary starting point dictated by the development process.
The notion of writers doing testing and developers drafting doc may not strike everyone as the most efficient use of resources, but lean stresses evenness and flow over the optimization of individual parts of the process, so it may well be an effective part of an overall optimal process for some organizations.
Hi Mark,
An excellent article and if I may add it completely lists the issues we face when trying to implement agile in documentation.
Writers in most offices are expected to work with multiple scrum teams and expecting a writer to be a part of each team’s meetings from planning to the sprint demo takes up most of the writer’s time and seems counterproductive. Very often the discussions in these meetings could be purely technical in nature and have no inputs for the writer.
Let’s hope Lean has more for documentation. Since we are an active part of software development, I feel any method that is implemented should consider aspects that will actually improve the way documentation development works.
Regards,
Sangeeta
Thanks for the comment, Sangeeta.
Historically, it has been very difficult to persuade development to change their process in order to make like easier for other parts of the organization. The lean principles of evenness and flow suggest that development should consider the impact of their practices on the other process, rather than optimizing purely for development efficiency. It may be hard work to drive that argument from the docs group, but when we try, I think it is important to frame the argument in terms of making the overall process better, rather than making life easier for docs. That way we can find allies in other departments that can help make the case for an overall lean approach.
When agile development starting gaining momentum, I pushed to get involved much earlier in the development process, going so far as to sit in during discussions among the sales and development teams, and the customers. Talk wasn’t enough; I had to show them. Once they see it in action and the end result, most get on board. I’ve been writing the agile way for over seven years now and have perfected my processes so that when the product is ready, the doc is ready.
Thanks for the comment, Lisa.
Indeed, in this as in all things, people need to see to believe. One of the things I like about the best books on Lean and Agile is that they deliver their exposition by way of storytelling. Not only do the stories stick in the mind, they are much more convincing than the mere statement of principles.
I think so often movements like these lose momentum when people start to advocate for them based on principles alone, and stop telling the stories. Not only is this less convincing, it also fails to get the real message across, meaning that people do not implement the new system correctly (and thus do not see the benefits).
I think this has happened to minimalism in tech comm. It has been boiled down to a set of principles which people routinely misunderstand and misapply because they are not told the original stories of the experiments on which minimalism was based. I wrote about this recently: /2013/07/02/what-is-minimalism/
Pingback: Tech Writer This Week for December 12, 2013 | TechWhirl