Agile Journey Index – A “Balanced” Guide for Continuous Improvement

If you were to have asked me about five years ago about agile team and organizational assessments, you might have gotten your head bit off. You see I used to be violently opposed to formally assessing agile teams in any way.

The roots of it probably related to aggregating team velocity. If you’re wondering, that’s not such a good thing to do either. I was worried about teams comparing themselves to each other and creating unhealthy or dysfunctional behaviors. I also worried about what THEY (leadership, managers, Project Managers, HR folks, etc.) would do with the information.

Now I’ve always felt that having maturity data around, in some form, was helpful to seasoned agile coaches. It just gets hairy when you start using it for organizational and x-team metrics. And it’s the inherent “metrics dysfunction” that is always lurking in the shadows this is a real concern.

Agile Assessments

But as time has passed, I’ve warmed up to assessment tools – or at least the notion of performing assessments. I think the key contributor to my growth in this area is Agile Bill Krebs. Bill is the creator of the Agile Journey Index, which is the assessment tool I use in my own coaching. Bill has shown me that some assessment tools can be used in a healthy way within agile transformations. That it’s not just about the tool, but it’s how you approach using it, communicating the results, and inspiring continuous improvement.

But before I get into the AJI, there are others on the market and the list is growing…

Agile Assessment Tools

I recently noticed that Sally Elatta and Agile Transformations have released their agile assessment tool – AgilityHealth. It reminded me that this space is getting a bit crowded as well (and probably will continue to expand) as folks figure out how to monetize assessments and their supporting tools.

Already in this space are:

• Evidence Based Management – Schwaber;
• Comparative Agility – Cohn;
• Forrester;
• AgileRBI – DavisBase;
• Scaled Agile Framework assessments – Leffingwell; and
• Agile Adoption Index – Sidky.

So just like agile certifications and scaling methods, the field of agile assessments is apparently alive and well…and growing. That growth is what has inspired me to share on AJI in the hope of introducing you to what I think is less well known, but congruent and effective approach to agile assessments.

Agile Journey Index

The Agile Journey Index (AJI) looks at agile practices in three, broad-brush categories:

1. Plan
2. Do
3. Check

Within each category, there are specific tactics, techniques, or practices that are evaluated. Let’s expand each category:

1. Plan
   1. User Stories
   2. Product Backlog
   3. Estimation
   4. Release Plan
   5. Iteration Plan
   6. Big Picture
   7. Governance
2. Do
   1. Stand-up Meeting
   2. Task Board
   3. Burndown
   4. Code Review
   5. Unit Test
   6. QA automation
   7. Quality Engineering
   8. Continuous Integration
   9. Done
3. Check
   1. Demo
   2. Retrospective
   3. Kaizen

The rating scale is from 1 (new) to 9 (experienced) for all 19 of the reviewed practices or activities. Getting a score of 10 is a special case that results from a peer / coach reviewing your “findings”.

The AJI provides evaluation examples for each, so the reviewer or assessor can ascertain the level of performance for each. But remember, and Bill makes this quite clear, these are subjective and context-based measures or evaluations.

Gamification

One of the primary reasons I like the AJI is that Bill has combined it with gamification. That is, assessments lead to teams gaining “Kaji Badges” that indicates their level of performance, which inspires good-natured and healthy competition across teams.

He works very hard to emphasize this positive interaction aspect that he wants the tool to produce and the badges help reinforce that. Reporting or x-team consolidation also is represented by badge levels of performance, which augment the “raw numbers”.

Assessment Dynamics

An AJI assessment is a periodic activity – often done quarterly or semi-annually. The assessment is mostly based on interviewing team members and having them tell stories and share examples along the boundaries of the 19 core practices.

The other side of the assessment is observation. So it’s a combination of the two. When I do an assessment, I like to get a broader brush view than simply gathering data the team I’m reviewing. Often I’ll go to leadership, other teams, and even customers to get an “external” perspective. Then I’ll contrast that against what the team is saying. The third component is my own direct observation. So to sum that up:

1. Team interviewing
2. Expert (coach-level) observation
3. Outside interviewing (x-team, leadership, and customer perceptions)

The assessment is an aggregation of all of the feedback into the AJI framework. In fact, the framework provides a nice model for all of the discussion and aggregation. I can’t imagine doing it without some sort of baseline tool to use.

The Point

From my perspective, and this is why I like the name Agile Journey Index, the entire point of the assessment is not to grade or compare cross-team performance. Remember, we are not grading teams for performance evaluation, monetary compensation or any other carrot & stick motivations. It’s to provide each team an assessment of their maturity and practices in a holistic and balanced fashion so that they can plot and plan their own personal improvement. Their agile journey if you will.

It’s mostly FOR the team.

Outsourced and Distributed Teams

Another useful part of assessments is integrating outsourced or distributed teams. I work for Velocity Partners, which is a leading nearshore outsource partner in Latin America. Nearly all of our client teams are “split” across the client and our remote teams or distributed in some way. Often the client teams and our teams have very different experience and approaches toward agile delivery.

One way to vet these differences and to integrate the two teams is to assess each of our capabilities relative to each other. Then we can discuss merging the approaches in a way that actuates the teams’ potential. We have something we call an Agile Alignment Workshop that I usually facilitate as a coach that works to align our clients and us.

And I’m not talking about adopting the very same approach or trying to create shared experience, which is often impossible. But if you simply expose the differences and discuss them, the teams can creatively adopt a shared style for operation and delivery. Assessment tooling gives us the platform and the language though for these conversations leading towards practical alignment.

The other valuable part is for our internal use. Our clients often engage us because we’re expert in agile approaches. One reason for that is our experience. But the other reason is the pressure that being a nearshore vendor places on us. You see, successfully “being Agile” is hard with a co-located team, but even more challenging in a distributed context. So we have to truly commit to the agile principles in order to deliver for our clients.

So assessing our capabilities can also serve to “show off” our strengths to our clients and very often they look to us for guidance and leadership in what are the best approaches to agile delivery and how to execute them effectively.

Scrum Product Ownership – Assessment

Even more to the point, I like the AJI so much that I “extended” it conceptually to include a deeper dive assessment into the maturity of your product ownership practices. I included this in my second edition of Scrum Product Ownership, in Appendix D.

Again, I don’t intend for this to be used as an outside-in organizational assessment. Instead, I am hoping that Product Owners and organizations will use it as a maturation benchmark and a means of focusing their own private learning’s.

Wrapping Up

What’s also nice about the Agile Journey Index is that Bill isn’t looking to attach certifications or generate immense revenue from it. He’s essentially “giving it away” to the agile community via Creative Commons licensing. Yes, he does ask that you recognize him as the source and that you implement the tool thoughtfully in the way it’s intended. And he does offer consulting and a bit of training around the tool. But that being said, he isn’t marketing it as, for example the AgilityHealth, Forrester, EBM, or AgileRBI products have been.

And another difference in the tool is that Bill has used it at IBM and AllScripts directly in his personal coaching. I’ve been using it for 2-3 years in my own coaching and many others have been leveraging it as well. So the point is that it is grounded in the “real world” and its been proven to help create an environment for evolving high-performing agile teams.

I hope this article has opened your eyes a bit to agile assessments and inspired you to take a look at the AJI and my product ownership extensions.

Stay agile my friends,
Bob.

Don’t forget to leave your comments below.

References
W. Krebs, P. Morgan, R. Ashton. The Agile Journey Index, 2012, http://www.agiledimensions.com
Kaji badges(tm) is a trademark of Agile Dimensions LLC and (c) 2012 Agile Dimensions LLC. If your organization would like issue formally certified badges contact Bill Krebs to have the facilitation course overview.