Saturday, June 15, 2013

Right-sizing Documentation in an Agile World

Within an Agile context, there is a focus on right-sizing documentation.  This is a response to the bureaucracy that has led to large volumes of project and product documents which are often left unread, are out-of-date the moment they are complete, and seem to have little value to the team.   On the other hand, I have heard folks who are in an Agile context say that no documentation is needed.  The truth is somewhere in between.    
One of the values in Agile is “working software over comprehensive documentation”.  This does not mean that there is no value to the item on the right, but there is more value to the item on the left.  The key is right-sizing the documentation.  Right-sizing means that the level of effort applied to write and maintain the documentation plus the value of that written document should have a greater return on investment (ROI) then not having that information readily available (i.e., the effort it would take reconstruct the information and impact of not having that information for current decisions).   With that in mind, here is some high-level guidance on right-sizing documentation. 
  • Documentation should be living and evolutionary and not “sitting on the shelf”.   Move away from the notion that document is “final”.
  • Consider some documentation to live at the product level where it may naturally evolve from both sprint to sprint and from release to release. 
  • Documentation should take on a collaborative nature.  It should not be written by one person to perfection, and then shared with others.  It should instead be shared often during draft to gain input   
  • Update your documentation continuously as new things are learned.  If you learn something that can impact the team or help with decision-making, go to the document and update it. 
  • Focus on just barely good enough documentation and avoid big upfront details which typically means a lot of guessing and wasted time.   Barely good enough means document what you currently know.  If you no longer need the document, it is reasonable to retire it. 
  • Be concise and lean.  Write in succinct prose and add bullet points if reasonable.  Avoid long and flowing prose which may lose the reader’s attention.
  • Documentation can take many forms.  It is not only a Word document, documentation can live on a wiki, in the Agile planning tool, as comments in code, and much more
  • Document decisions.  Knowing the decisions helps those who must maintain the product and understand why things were decided or done in a certain way.   
  • Realize that there is a difference between internal and external documentation.  External documents may be a User Guide or document that gets delivered to the customer.  These should be considered as “working software” and follow the rules of the done criteria.    

What might get documented in an Agile world?  The major documentation comes in the forms of user stories and epics.  This includes writing the user story in canonical form, with details, and acceptance criteria.  Technically, another significant form of documentation is the code itself. It should be written in team agreed coding standards with appropriate comments that specify what the code does (or simply and clearly enough that don't need explanation) and why it was written a certain why.  Remember that code needs to be maintained so if someone has to "figure out" what you wrote and why in order to support the code, then it is problematic.

You should consider lean versions of your architecture, coding, and test strategies that help inform the team of ways of working. Impediments that require effort to fix should be documented.  Items such as coding standards, an overview of your working processes and policies, done criteria such also be documented and shared. 

What do you think of this guidance?  Do they make sense?  Do you have any other tips?


  1. My experience — programs become documents and documents become programs. Parts of software solution, namely executable process diagrams, become illustrations in the documentation. Classic documents, namely forms, become interactive services.


  2. You have made it quite clear. One should avoid both extremes: 'No documentation' vs 'Comprehensive, very elaborate documentation'. The intelligence is to read in-between the lines. I think Sprint Zero may be used to decide the level of documentation needed. This step is very essential for Life Science projects where FDA regulation (section 21 CFR Part 11) are mandatory. There is some sort of compliance factor in other industries, too.
    Suhas Tare

  3. Thank you for your input Alexander and Suhas!

  4. I absolutely agree that there must be a balance. I've typically found that there are 3 types of documentation:
    - persistent, living documents (requirements, tests, very high-level architectural descriptions) that provide enough value to be maintained going forward
    - transient documents that are useful in the context of a given release, worthwhile to save as reference to your mindset at the time, but not worth the effort to maintain going forward
    - documents that can be generated directly from the code, database, etc.

    Avoiding putting effort into the last two (in my experience) goes a long way toward achieving the balance.

  5. This comment has been removed by the author.

  6. Hi Mario,
    Nice article. Good to have out there.
    In my experience the most important documents are three:
    1. architectural diagram maintained up to date, for new people;
    2. copious comments in code, essential for maintenance, and enforced through code reviews
    3. intelligent test documentation (no repeating the same thing over and over), kept up to date, reviewed in the same strict manner as code, and kept in the same CM directory with the code, or pointed to from there.

    Warm regards, Marina Shalmon
    Deep Roots Agile Coach

  7. Hi Mario,

    Just found this link on LinkedIn. I agree you need a balance and agree big documents written up-front and handed off are worthless. I also think comments in code are useful to developers later but you want something outside the code so if customers call tech support (or if in IT, the help desk) later there's a place to look up the answer without asking the developers to get involved.

    In my experience the best approach was when a tool like DOORs was integrated into the software process so that the requirements were there BUT it evolved during design and development to represent the final design (from a business requirements level although if the developers wanted a place to put additional info, that was the place). Then QA tested against it as part of the testing/DoD. This worked because each DOORs requirement was linked to our tracking tool. So the requirements or stories were in the tracker. The tracking tool had interactive comments and tracked the progress of the code up through code check-in and test. The tracker was also the bug tracker so everything was in one place instead of stories in one tool, bugs in another.

    Then the next release started with modifying the DOORs specs to add the new features and functionality in. DOORs could be baselined so you always had info about how the old versions worked but the latest said how the latest code worked. If a customer Call came in it was clear. If DOOR said the code should do (A) and the code didn't do (A), it was a bug. If the code matched the spec, the customer was asking for an enhancement request. If the spec didn't say anything about it but the request made business sense, it was a "gap". Helped with traceability and accountability.

    With Agile, the team moved away from DOORs and fell into disarray.

    We wrote a new tool to let the PO write the stories but organize them (like a spec) but in that way the stories are living breathing items. The tool also manages stories like a tracker - so stories, bugs, customer calls. Everything in one place. The tool founders aren't actively selling the tool but it's an interesting concept if you are interested.

  8. Thank you for sharing very informatics and useful post about very useful info.... Turbo-IVP (Invoice Validation Portal) and Turbo eSigner