Is Agile Mainstream yet?

There continues to be a lot of debate on whether Agile is mainstream. According to a Forrester report published in early 2010, while widespread “Agile” use of the iterative software development processes is found, " teams are not adopting scrum, extreme programming, or another specific Agile approach, but are embracing agile as an ethos or philosophy and cherry-picking the best bits from many different process models to develop a formula unique to their own situation."  

However, the largest category in the survey – and the one that is the most telling is that 30.6% of the respondents said they do not use a formal process methodology.  Add to this my own experience implementing Agile, reading the latest Agile literature (e.g., articles, research, books, etc.), and discussing Agile (and Agile implementations) with people across numerous companies in North America, Europe, and Asia, and what this indicates to me is that:

• There is definitely broad awareness of Agile
• There are many companies who are on the Agile bandwagon because it is seems like the right thing to do
• There are many Agile “book-read” folks who have not really experienced an Agile implementation
• There are some teams who are “cherry-picking” parts of Agile process for their own Agile implementation
• There are fewer teams who are applying end-to-end Agile methods and practices across their lifecycle
• The companies that have made the cultural shift to the Agile mindset are still a minority

The question becomes, does this really represent a pervasive enough understanding of Agile and a thorough enough adoption of Agile across the industry for it to be mainstream? 

IMHO, the answer is not yet.  The reasons are that I am not sure if companies have fully "realized" what Agile is and how to implement it.  An indicator is whether enough people or teams who have implemented Agile can recognize common steps to a successful Agile implementation.  Another indicator is whether those that have implemented Agile have actually made the cultural shift (aka, Agile mindset or self-empowered teams, servant-leader mentality, etc. ) in order to gain the benefits of Agile and to make it mainstream?

So what do you think? 

Holistic view of Continuous Integration & Build (elements of CIB) – Part 2 of 9

In the last episode (e.g., 1 of 9) of this series, I introduced the topic of Continuous Integration and Build (CIB). In this episode, I will introduce the elements of CIB. The key elements involve four areas: a mindset change to thinking more continuously; the entrance criteria that makes the CIB process perform; the components to initiate an effective CIB process; and finally key infrastructure to enable the CIB process. Let us walk through these areas at a high-level.

An Agile mindset merges with the CM mindset. 

• A very interesting cultural shift occurs when the concept of “continuous” is ingrained in the culture and method. Agile embraces a mindset of continuous change where the build process moves from an event-based integration process to a continuous integration process. In other words, no one needs to hold onto large amounts of changes for a major integration effort or declare that builds will occur nightly, weekly, even hourly.

The entrance criteria for an effective and lean continuous integration and build process include:

• The ability to specify the right ‘bite-size’ level of story or requirements tasks that represent changes that allow for granular and frequent code changes. This implies that the Agile team can understand the stories well enough to divide them up in small and consumable tasks which allow the programmer to make changes frequently and incrementally.

The key components to initiate an effective continuous integration and build process include:

• Right-size the branching strategy that reduces risk yet ensures code stability where people can work in a stable workspace without being impact by changes of others on a regular basis.
• Shift in roles and responsibilities of who performs merging and building.
• Minimize the merge process to reduce work for development
• Emphasize building in general and understanding the build levels so it is clear who the target of the build is for. Builds can occur within a private workspace and within shared branches like the mainline or project branches.
• Test with teeth by establishing and conducting unit testing at the individual programmer level and then smoke test after the integration build.

Underneath all of this, there is a need for infrastructure to support a continuous integration and build process. The two primary elements of this include:

• CM version control system that has the capability of establishing the desired branching strategy, has an automated and intelligent merging capability, and can integrate with continuous integration and build tools.
• Continuous integration and build tool that supports an automated build process. There are many continuous integration and build tools on the marketing ranging from vendor products to open source and freeware tools.

Let’s delve deeper into each area. Consider reading the next episode which focuses on the Agile and CM Mindsets (Part 3 of 9) and what this means in an Continuous Integration and Build process.

Note: If you started with this entry (Part 2), consider reading part 1, Holistic view of Continuous Integration & Build – Part 1 of 9

Holistic view of Continuous Integration & Build – Part 1 of 9

As you may know, the term CIB (or continuous integration and build) refers to the process of integrating code frequently (or on-demand) to reduce large integrations, complexity, and pain in the future and to make functional software readily available for testing and the customer. It provides development with immediate feedback on the success or failure of changes via an integration, full build, and smoke test of the product and reduces large integration efforts.

Continuous integration and build moves the build process from an event-based integration process to a continuous integration process. This then moves us away from the infrequent and often painful integrations (a.k.a., merges) and move to a continuous integration and build process that becomes part of the team’s daily activities. This also marks a significant increase in check-outs, merges, check-ins, and builds.

The benefits of CIB include:

• Integration ensures the integrity of your code baseline.
• Building frequently lets you and the customer know where things stand as a mark of value delivered.
• Continuous integration raises merging issues to the forefront more quickly for more expedient resolution.
• Frequent builds have fewer changed files which reduces the amount of time on debugging build issues and painful integrations.
• Working functionality provides continuous feedback.
• Can be applied to any project methodology (Agile, Waterfall, Hybrid, etc.)

The challenges of CIB include:

• It is often easier said than done
• It requires a focus on areas broader than most folks realize (beyond just merge and build)
• May cause more churn than progress if not managed well
• Requires a mindset change, thinking in smaller units, bite-size tasks and continuous change
• Moves from event-driven model with ceremony to a continuous change model
• Adds stress and load to CM and build tools

So what are some key elements of focus for CIB?  Learn about these elements in part 2 of the CIB series on Elements of Continuous Integration and Build.

The Knight bringing Agile to the Day

Once upon a time, a Knight challenged the King saying that we should provide people with what they need and not what we want to provide them. Instead of asking people for all of their needs now and not deliver until a year later, we should deliver their more important needs in shorter time periods to ensure we provide them with their needs sooner and then allow them to adapt to their needs as life changes around them.

The Knight learned that the marketplace and the customers therein drove the real needs. This gets to the heart of providing business value, value that the customer perceives, value that can change in this ever-changing world.

As the Agile Manifesto (the Knight's creed) says, Agile values working software over comprehensive documentation. Working software is where the customer sees the value. The “right” amount of documentation, neither too comprehensive nor too little, can lead us to working software more quickly.

Individuals and interactions have more value than processes and tools. This does not mean that processes and tools are not important, it is just that defined processes and tools should not determine how the individuals should interact to get their work done.

Customer collaboration is valued over contract negotiations since Agile values the continuous interaction with customers to ensure we are constantly reducing the risk and increasing the certainty of delivering what the customer really needs.

And finally, responding to change over following a plan allows us to adapt to change with collaborative control that ensures the change is both welcome, understood, and continuously validated.

Agile embraces change and accepts the fact that life is uncertain. By providing methods and techniques to minimize risk and increase certainty, this ensures we close the gap between what the customer actually wants and what we end up delivering.

With that, the Knight brought Agile into the day and people into the light.

Cloudy with a chance of Agility - CM predictions for 2010

As we gaze in the horizon, what do we think will be hot in the CM landscape and improve our working lives? What might be some of the latest trends in the industry? My prediction is the CM weather report will be “Cloudy with a chance of Agility”. My predictions will focus on:

• Agile in the forefront of CM
• Extending the CM reach into ALM and beyond
• CM “in the Clouds”

Prediction #1: Agile in the forefront of CM

I predict that we will continue to see a stronger focus on agility in the way we approach and deploy CM. This is because Agile methods are continuing to increase in adoption. With the need to adapt and change, comes the need for CM that is both lean, yet well integrated to support the Agile processes. We'll see CM tools cater more to Agile. We'll see more publications focusing on CM and Agile.

Prediction #2: Extending the CM reach into ALM and beyond
As we continue into the future, we will see CM extending into the ALM space and then see ALM extended into a more unified approach. Integration across engineering areas helps teams streamline their processes and reduces the effort of implementation and maintenance of manual integrations. We'll see Agile ALM solutions with CM as its core for organizations looking to improve and scale their Agile processes while still maintaining control. And we may see more holistic unified change and release management solutions

Prediction #3: CM “in the Clouds”
As we looking into 2010 and the future, there may be two focus areas relating to the “Cloud”. The first is ensuring that there is configuration management of the Cloud infrastructure and the second is that there will be more of a focus on hosted CM services in the clouds. Companies are looking for software as a service (SAAS) solutions to limit their infrastructure debt, but these environments also require solid configuration management so the customers of the SAAS solutions can feel confident that changes within these environments will be effectively managed. The second is providing a CM service for software development in the clouds. This way development teams do not need to incur the cost and effort of setting up a CM environment and maintaining it, but instead use a CM environment in the clouds with built in version control, build management, and defect tracking tools that already exist.

Summary
As we look into 2010, conditions may get cloudy but not in the meteorological sense. Companies are looking for “software as a service” (SAAS) solutions to limit their technology debt, but will want to ensure the infrastructure is well managed. Agility will continue to show up in various forms in both the Configuration Management (CM) and Application Lifecycle Management (ALM) contexts. Also, books such as “Adapting Configuration Management for Agile Teams” and blogs will help CM and Agile teams understand and adapt to Agile methods. Whether your forecast is sunny or cloudy (or both), consider flexibility, adaptability, and agility in driving your business! Have a productive 2010!!!

To read the full article on this, feel free to visit: http://www.cmcrossroads.com/cm-journal-articles/13187-cloudy-with-a-chance-of-agility-cm-predictions-for-2010

What are your thoughts on where CM is moving to in 2010? 

Book released in the US - Adapting Configuration Management for Agile Teams

Exciting news! My book “Adapting Configuration Management for Agile Teams” which is on the world market is available in paperback and now as a Kindle ebook!  It is the first book of its kind that combines discussions on both CM and Agile. Agile is the facilitator of change while CM is the enabler of change. When done properly, CM provides the right level of control for Agile projects to maintain and increase its high level of velocity. CM for Agile can be a very powerful combination when done effectively.

How can it help you? Aimed at all levels of professionals in the Agile, Configuration Management (CM), Project Management, Product Management, and Development fields, this book provides very tangible support for Agile teams. It focuses on how CM with its practices and infrastructure can be adapted and managed in order to directly benefit Agile projects

What are some unique features of this book that can help you?

 • Provides both an Agile Primer and a CM Primer. For those experienced with CM, it provides a solid overview on Agile(or visa-versa on CM) so you can come up to speed. It can be great for someone who wants to learn both. It’s almost like a two-for-the-price-of-one book!

 • Provides insightful and concrete guidance on adapting CM for Agile methods while maintaining CM principles and allowing minimal structure needed for Agile projects to continue to run fast. Adapting CM for Agile doesn't mean throwing out the CM principles of identification, control, audit and report. Instead it means that CM should be adapted to allow the lean nature of Agile. The question is how do we do this? Inside, learn how to adapt:

o Continuous Integration and Build applying notions of bit-size tasks, right-sizing branches, minimizing merging, emphasizing continuous build, and testing with teeth.
o CM Planning (aka CM Vision) moving away from big-effort-up-front (BEUF) to a more evolutionary approach, applying an iteration 0 to get started, adapting CM roles and responsibilities, and using wikis and other CM formats.
o CM to support more effective Refactoring, Pair Programming, and Test Driven Development (TDD), Traceability, Baselines, Problem Management, CM Audit, and CM Reporting and lean metrics.
o CM support for Distributed Development for Agile teams when co-location is not an option by performing a distributed analysis and considering code access solutions, and supporting a Cloud infrastructure option.
o CM infrastructure for Agile thru implementing CM Envisioning for a brand new product following Agile methods or thru CM Refactoring to adapt CM for an existing product line moving to Agile methods.

• Will help you understand how the more modern CM features help with implementing Agile in an effective manner and that CM tools are a strategic Agile partner.
• Will help you understand how to more effectively support Agile when your organization, product, or project must align with standards and frameworks.

Again the book is called “Adapting Configuration Management for Agile Teams” (ISBN-10: 0470746637). This book is available on both the Agile and Wiley websites. Also, if you want the original CM book which people love as a great CM reference book, feel free to consider, “Software Configuration Management Implementation Roadmap” (ISBN-10: 0470862645).

Have a great day!

How Valuable is Configuration Management to Agile Professionals?

Being a strong proponent of both Agile and Configuration Management (CM), I was curious to learn if other Agile professionals perceived CM to be of value in an Agile setting. Having worked in both areas for some time, my initial hypothesis was "yes" Agile professionals do value CM. Having recently attended the "Give thanks for Scrum" event, both Ken Schwaber and Jeff Sutherland referenced the importance of builds and control to Agile (a CM component).

Having worked in Agile teams and hearing stories from Agile and CM professionals, I decided to exercise my hypothesis and find out if in fact Agile professionals do find CM tools and processes to be of value in their software development work and at what level. With that in mind, I conducted two studies to determine the importance of CM.

The first study was conducted in both the “Agile Journal” and on “Linkedin” that focused on the importance of CM tools (e.g., version control) for Agile projects. it is interesting to note that this appears to be the first study of its kind. I provided five categories as possible answers to the question: Extremely Important; Very Important; Somewhat Important; Not very Important; and Not Important at all.

The results from this study (based on 208 respondents) was very telling and indicate that CM tools (or at least the version control function) are indeed important for Agile teams working in software development. When combining the number of respondents who answered either “Extremely Important” or “Very Important”, the result was a dramatic 94%. This is quite an amazing number.

This means that 75% of the respondents answering “Extremely Important” and 19% answering “Very Important”. I wonder if there are any other tools where 75% of Agile professionals would answer as being extremely important to their work. Respondents who answered “Not Very Important” or “Not Important at all” accounted for only 2%. While there were over 200 people who responded, I do not consider this conclusive data. However, it is certainly data to “chew on”.

The second study sought to drill down further as to which CM practices are perceived to be valuable by Agile professionals and which are more valuable than others. With this in mind, I conducted a second survey in the “Agile Journal” where I asked Agile professionals within a polling questionnaire to please place a value rating on each of the following CM functional areas:

• CM Planning (CM for the project including planning activities, roles, etc.)
• Version Control (tool with Check-out/check-in process)
• Build Management (tool with continuous build process)
• Branching and Merging (tool and continuous integration process)
• Change Control (process of managing changes)
• Problem Management (tool and process – a.k.a., defect tracking, issue mgt)
• Audit (process – baseline, etc.)
• Reporting/Status Accounting (process)
• Release Engineering (tool and process – a.k.a., migration and installation)

I provided 10 value choices (from 1 to 10) where 1 is the lowest value and 10 is the highest value. While not meant to be an exact survey, it provided some interesting data.

The results from this study (based on a modest 25 respondents) indicate a range of perceived values for the various CM functions. Build Management received the overall highest average value score with 9.21 (out of 10). This is not surprising since continuous integration and build is a highly valued practice for those who implement Agile. This was closely followed by Version Control which received an average value score of 9.13. This may be because of the recognized need of control due to the high volume of code changes. The high value of the version control function correlates well with the first study where a high percentage of Agile professionals indicated that CM tools (e.g., version control) were either extremely important or very important.

Branching and Merging is next with a score of 8.33. When considering that a big part of continuous integration assumes merging, then this high score is not surprising. Most interesting is Change Control with the fourth highest score of 7.92. In the survey, there was a short description included with Change Control that said “process of managing changes”. With this additional piece of information, it may be that Agile professionals align control of changes with iteration planning where change is periodically managed. While I do not consider this conclusive data, it is certainly data to “chew on”.

The conclusion is Configuration Management (CM) appears to be valued by Agile professionals after receiving a combined score of 94% who responded with either “Extremely Important” or “Very Important” from the first survey. Why this is may be because Agile is a facilitator of change that requires change mechanisms that can handle the high volume of change. CM provides those change mechanisms to ensure the integrity of change on a software project. Being a strong proponent of both Configuration Management and Agile, I am happy to see these results. For Agile professionals who have not yet considered CM as a valuable asset to their work, it may be time to do so now. It may be to your advantage. For Agile professionals who do understand this powerful combination, more power to you and it may be worth giving some of the CM functions a second look.

To see the full study on this topic, feel free to read the article the "Value of Configuration Management by Agile Professionals" found in the November 2009 edition of the Agile Journal found at: http://www.agilejournal.com/articles/columns/column-articles/2506-value-of-configuration-management-by-agile-professionals