Are there Dangers with ChatGPT for Agile?

How will ChatGPT impact Agile? This article discusses ChatGPT and its implications to Agile in the industry today. ChatGPT is taking the internet by storm and hard to ignore. Because of this, it cannot be ignored by those in the Agile field. What are the implications of ChatGPT on Agile? Here is a brief summary of what is ChatGPT and a review of what is Agile and its current journey.

What is ChatGPT?

ChatGPT is an artificial intelligence (AI) chatbot-type tool developed by OpenAI. It is adept at producing human text-based output on the input it is given. This model incorporates a large body of text data and can create responses to questions, write articles, and more. The challenges with ChatGPT are that it is only as good as the “large body of text data”, can be used maliciously and with bias, can spread misinformation, and is ethically complex in its application and future application. This applies to any field that people may use it for including Agile.   

What is Agile?

Once upon a time (in 2001) Agile was unveiled based on the Manifesto of Agile Software Development which is comprised of Agile Values and Principles. The objective of articulating the values and principles is to apply them in the form of an Agile transformation to derive better business results. However, the manifesto does not provide guidance on how to apply Agile. 

Soon, a number of processes and methods (e.g., XP, Scrum, Kanban, SAFe, etc.) were established to construct and apply agile ways of working. Agile has also spawn a number of certification programs in an attempt to educate people in Agile ways of working, in some cases aligned with a process or method. During this same time, Agile coaches were educated to help their own companies and Agile consultancies to help other companies apply Agile ways of working. The Agile movement has grown and expanded in a number of fields beyond software development.  After over 20 years, what are the results? The challenges are three-fold.  

• First, the current state of Agile is underwhelming. The most recent State of Agile Report (16th Annual – 2023), tells us the following. Only 18 percent of organizations implemented Agile for all the teams. Around 50 percent of respondents report that less than half of their teams are using agile, and 84 percent acknowledge that their organizations are below a high level of competencies. There is clearly plenty of opportunity for growth.
• Second, some of the Agile savvy (e.g., coaches, consultants, leaders, managers) seemed to lack an understanding of what is agile. In an Agile study where 109 agile professionals answered a survey on Scrum events and Agile principles, 59% could name 3 or more of the five Scrum events, while only 11% knew 3 or more of the twelve Agile principles. This is quite astounding. And they didn’t need the full statement of the principle but got credit for even the key words of the principle. The concluding hypothesis is that the reason there is such a lack of awareness of Agile principles is that there is much less focused on the mindset and culture and maybe too much focus on the mechanics.
• Third, the implementation of an Agile transformation is complex per the definition provided by the Cynefin framework. Agile transformations are neither linear nor predictive. It depends on the readiness of the culture and willingness of its leaders in their ability to move forward. Complexity means that it is not clear on what the best next step is until you act, ergo you need to probe, sense, response your way forward. This is why experimentation helps reveal what is possible each step of the way. You must both meet the company and teams where they are and help them determine what is the next step to further the transformation.  

What this tells us is that there are great opportunities for improvement and that there is no easy way to apply Agile, no one-size fits all, and no clear roadmap. Why? Because every organization is different due to their current culture, size, fields, practices, and more. 

Implication of ChatGPT and Agile

Now that we have an overview of both topics, the question is what are the implications of ChatGPT to Agile (and vica-versa)?  I’ll start by saying “What you put in is what you get out”. ChatGPT is only as good as the “large body of text data” available to pull from. The good news is that today there are reams of text data on Agile. The bad news is that there is no rating system on the quality of most of the Agile related information. With the advent of blog’s, there is a large body of unverified knowledge that enters into the “large body” of available data. What are the implications of this? 

• Arguable Quality of response - The quality of ChatGPT generated articles and answers should be read with a grain of salt. This isn’t a “knock” on ChatGPT, and instead it is due to the quality of the body of text data that ChatGPT draws from. And the reality is there is no one right way of applying Agile.  
• Propensity for Misinformation - There is a danger of misinformation and abuse of those who use ChatGPT to bias their responses. Some may be accidental as the body of text being pulled in isn’t broadly approved or agreed upon. While I don’t expect that most will be intentionally abusive, do keep in mind, there is money to be made in selling agile so bias may be seen.  
• Not doing your own Research - While you may want to occasionally use ChatGPT, it is better to learn from the body of Agile knowledge out there (e.g., books, articles, presentations, seminars, etc.) according to the areas that will benefit your current needs in your Agile transformation or need. In other words, do your own research so you can critically judge the quality of information that gets generated.
• Taking Agile Jobs - Can ChatGPT take jobs away from Agile Coaches and Consultants? This is unlikely as a significant part of an Agile transformation include coaches and consultants who have been on a transformation journey that can help companies navigate the complexity of both the current needs and the anticipation of near-term needs. ChatGPT cannot “read the room” like an Agile Coach. Should a company think that ChatGPT will be “enough”, it highlights that they don’t understand the complexities of a transformation and what it takes to change culture.

Summation

Now that you have some background, let us again turn to the question, “how will ChatGPT impact Agile?” There will be those that use ChatGPT to provide answers for Agile theory and questions. If you want to write an Agile article, it will help provide input and insight, although you have to be aware that the value of the information is only as good as what it pulls from. Think of ChatGPT as another resource to help you think through your ideas on agile topics and how it may help you in your Agile transformation. However, just remember, it is just a tool like other tools.     

It is unlikely that ChatGPT will take over Agile roles and the art of the transformation. A big part of Agile transformation is discovering, observing, and experimenting on what will work and what will make progress. Remember, when defining Agile, it really implies a transformation. This is a combination of doing agile and more importantly being agile. This means transforming mindset and culture. It is currently unlikely that ChatGPT will have this capability as transformations are complex with the real need to experiment (e.g., probe, sense, response) toward progress.  Coaches and consultants are still important to help transform organizations and more importantly to help leaders and teams make the mindset-shift to truly becoming Agile. 

Breaking Bad Story Point Anti-Patterns

There is occasional controversy in the Agile coaching community on whether to apply Story Points on user stories. This often stems from the challenge where some think that story points are a direct replacement for estimation and an expectation of precision while others suggest that instead of story point sizing, we should focus on focusing on small pieces of work. In other words, when applying a practice (e.g., story points), a bad pattern is implemented which is opposed to the way practice is meant to be applied. This is known as an anti-pattern. Anti-patterns lead to results that are counterproductive to the intent of the practice and reduce their effectiveness. In other words, not good. 

What is Story Point Sizing? 

Before we go further, let’s take a moment to define what is a story point? A story point is a unit of measure describing the size, complexity, and risk to gauge the amount of work needed to complete a particular user story. 

Story points are also a relative measure to a specific team. Every team creates their own story point sizing framework based on the type of work they do, the skills and experience of the team, and what they personally perceive to be a small, medium, or large amount of work. The team collaboratively determines the story points for each story based on its perceived size, complexity, and risk. 

Story points are often correlated with small, medium, and large sized work. Some use the Fibonacci as it provides a numeric distribution that can be used to differentiate between sizes of work typically using 1, 2, 3, 5, 8, 13, 21. When using Fibonacci numbers, 1, 2, and 3 are aligned with small work, 5 and 8 are medium sized work, and 13 and 21 denote large pieces of work. 

Velocity is a metric that can help a team understand their sustainable pace by identifying the amount of story points of work a team can complete in an iteration. In each case (e.g., story points and velocity), this is why you should not compare the story point velocity across different teams. Story points and velocity are very specifically a team measure and should not be tampered with by those outside of the team. 

Anti-patterns of applying Story Points 

The reason there is concern surrounding applying story points is the ways it gets applied. As mentioned, story points are a combination of size, complexity, and risk of that specific piece of work (aka., user story). This means they are not meant to be used as a predictor or for accuracy. Here are several anti-patterns of story points in the way they get applied: 

• Unfit Translation anti-pattern - Some apply story point sizing as if it is nothing more than an estimate of hours and days (and weeks and months). There is often a direct translation of small, medium, and large to days. 
• Pretend Certainty anti-pattern - There is often ascribed a sense of certainty when applying story point sizing where it gets used to predict when work will get done. At best, it can help with understanding progress, but you would never estimate “all” of the work up front anyway as priorities (and requirements) tend to shift so it would be a waste to do so. 
• Pretend Precision anti-pattern – There is precision when using story points which isn’t appropriate. The numbers that story points represent are meant to be ball-park numbers as it is an amalgamation of size, complexity, and risk. 
• Contrived Comparison anti-pattern – Some organizations attempt to compare story points and velocity across teams even though they are relative to the team’s composition and the type of work they focus on. This is inappropriate and decreases the integrity of story points and velocity. 
• Effort Tampering anti-pattern – This occurs when someone outside of the team (usually management) attempts to influence the amount a work a team does by insisting on improvement. This impacts the integrity of the story point sizing framework and the velocity data as those are meant specifically for that team to have meaning. 
• Inflation anti-pattern – This can be the result of when someone outside of the team applies the Effort Tampering anti-pattern by attempts to make the team “work harder”. The result may be that the team inflates their numbers to ‘satisfy’ the influencer and effectively impact the integrity of the story point sizing framework. 
• Productivity anti-pattern – This is when story points and velocity get conflated as a productivity measure by those outside of the team. They are not productivity measures and will warp the intent of both story points and velocity. 

Mending Anti-Patterns  

The best way to eliminate or reduce anti-patterns, is to first understand what anti-patterns look like (see Anti-Pattern section above and search of other information on anti-patterns). Then do detective work to uncover what anti-patterns may exist. Follow this with determining an action to remove or eliminate them. You can do this through a theme-based retrospective where the focus is on identifying anti-patterns. 

Anti-patterns within an organization are more commonplace than you think. They are often due to a lack of clarity of what are story points, how they should be applied, and a lack of awareness that they are specifically a team-based measure. There may also be management or team member influence to use the practice or technique incorrectly. If too many of these anti-patterns exist, then the value of using story points as a team measure and as an instructive tool to help team gauge what is considered small diminishes.