Read my Lips There are NO Magic Numbers in Agile!

When I was a new software manager, which if I’m honest was more than a few years ago, one of the things I went looking for were magic numbers.

Let me share a few with you:

• What is the most effective ratio of Developers to Testers?
• If I was planning a software project, how long did it usually take to create the requirements? vs. writing the code? vs. doing the testing?
• How many architects does it typically take for a larger-scale software project?
• When I hire a new engineer, how long should it take for them to ramp up?
  • The answer is 3 months…Trust Me!
• When someone resigns, how long should it take (on average) to replace them?
• For every tester, for every developer, and for every team lead, what was the right “focus factor” to plan their weekly work efforts?
  • I can’t help myself…the right number is 70% or sometimes folks like to say 5.5 hours of work time per day…Trust Me!
• If I invested in automation, how many testers could I re-target per 1000 of automated test cases?
• How many projects could I split (share) across my 100-person development organization?
• How many resources (mostly I mean people) does it take to tackle a small project? A medium project? A large project?
• How many large projects could I run through my organization in parallel?
• I’m ramping up a new company with some new “Agile” projects. How fast can I hire folks? How fast can I pull agile teams together? How fast can they be fully operational?
• How many bugs, on average will I find for every 1000 lines of code? And what is the average time to fix those bugs?
• What is the right amount of the backlog to reserve for technical debt?
• How about the % of coverage for: Unit tests? Automation? Regression?
  • I can help myself again…it’s 80% for unit and 100% for everything else…Trust Me!
• What’s the right balance between test automation and manual test cases?
• 2 weeks is the perfect sprint length, right?
• We don’t have the money to hire a Scrum Master per team. Is it a full-time job? It can’t be. How many teams can a Scrum Master effectively handle? Please tell me it’s 5!
• I have 1000 user stories on our Product Backlog…is that too many? Or too few? What is the right number?
  • The answer is – 2 releases worth of PBI’s or ~250 stories…Trust Me!
• Scrum says that teams should be 7 +/- 2 in size. That means I can’t have a 10-person Scrum team—right?

Well, that may have been more than a “few”. Did this list help you to understand what I mean by magic numbers?

Why the Focus?

Somewhere back in history, software project managers and leaders allowed our “engineering heads” to take over in our planning. We discovered that we could pull spreadsheets together to plan, monitor, and predict almost any aspect of software development.

Microsoft Project and Excel then are the primary culprits in this endeavor. We felt, and it felt good I might add, that we can take all of the complexity and variability and thinking behind the creation of software and model it arithmetically.

How cool is that?

We could also remove context-driven thinking from the equation – especially since the numbers hold true no matter the context. We can simply plug them in…and away we go. I often think of their use as being the easy way out or a silver bullet. While certainly easy, I’ve found that there are no silver bullets and we were nearly always wrong. How about you?

Harsh Reality

But the reality is software projects (people, complexity, learning, creativity) cannot be modeled, estimated, or precisely measured.

Every team and project is different and you should stay away from using any magic numbers as replacements for your own thinking, observations, and ongoing learning about what works in your teams’ contexts.

Trying to find a hiring time factor and a new engineer ramp-up factor for your project costs versus impact forecasts is a fool’s errand. Sure, you can easily come up with some magic numbers and put the model together. But my experience is that it NEVER models the reality of your situation.

And the worst part is we usually convince ourselves that it does – constantly staring at them and tweaking our models constantly to “manage” the project.

Are They ALL Bad?

No, of course not. There’s usually some hard fought wisdom embedded within the numbers. And for certain contexts they effectively serve to establish an entry-level thinking model for our planning.

For example, I like using developer to tester ratios as a “health check” for agile teams I’m coaching. If I see a 6:0 or 10:1 ratio, then I’d probably consider that suspect and look to see if the poor tester is overloaded. So, in this case, the ratios aren’t fixed per team but are balance indicators.

I might add though that without the ratio awareness, I could also look at the teams’ velocity and output quality and determine the imbalance just as easily.

What to do Instead?

Look at your context. Ask your teams. And create your numbers from your history and your own experience…in your contexts.

But Bob, we’re starting a new set of teams – so I have no context. Yet, I need to pull a model and a plan together. My boss, the CTO, wants it by tomorrow.

My advice then would be to wait. Tell your managers and executives that you don’t know yet and that you’ll have to collect some real-time execution data from your teams. I know this won’t be that popular, but it IS the truth.

For example, my colleague Mary Thorn and I go round and round about developer-to-tester ratios in agile team contexts. Mary has a reference ratio of 3:1 if you’re doing more manual testing and 2:1 if you also want to develop automation in parallel with your functional testing. I always push back on Mary about even SHARING these numbers with people in classes and at conferences. I’m always afraid that they’ll blindly take her advice as gospel and rush back to their organizations to implement them. Many do.

But Mary is right that these ratios can be useful IF we view them and act on them in the right way.

There are other magic numbers that can be more harmful. For example, if your boss asks you to pull together a project level plan and date commitment for a set of 10 agile teams, then you’ll be challenged to guess at each teams’ ongoing velocity over time. If they are new teams, you’ll want to explore the ramp-up. And in order to provide a holistic date / scope commitment, you’ll want to aggregate their velocities into a single number. In this case I think you’re on very dangerous ground – more dangerous if these are new teams.

Why?

Because you’re guessing about the numbers, but you’re also making some sort of business commitment based on them. It would be much better to gather some execution based data across the teams and then do your forecasting based on real numbers versus magic numbers.

Wrapping Up

I actually misspoke in the title. We should get away from magic numbers in all of our software methods and approaches, not just in agile. They’re simply misleading at best and create dysfunctional organizations at worst.

Let me wrap up with one final story. For many years, I’ve coached new Scrum teams towards effectively starting up. I have a “recipe” for sprint planning for new teams.

1. Instead of allowing the organization to assign a per-day magic number for team members, I ask each individual team member to offer their capacity for sprint planning. I want them to consider their vacation time, other projects, meetings, bug fixes, interrupts, etc. and give me an estimate for the time they have FOR this teams’ sprint.
2. I also ask team members only to plan in ½ day increments. That would be defining their capacity in ½ day increments and tasking out the sprint work in ½ day increments.

I want to start them off avoiding hours and avoiding magic numbers. I want them thinking of their personal work capacity and committing to their work.

I usually get pushback from the leadership team with this approach. Telling me that testers should plan at 6.5 hours per day, developers at 7 hours per day, and team leads at 5.5 hours per day. They usually have names for these numbers – focus factor being a common one. While the numbers may generally be accurate or based on some good intent, I always ask to allow each team member to consider their context for the next sprint and to commit their own capacity.

What I always find is much more real world variance than the magic numbers accommodated. But beyond that, it gives the team a sense of empowerment and trust to plan and commit for themselves. It also turns out that they’re pretty good at it. Now how agile is that?

Stay agile my friends,
Bob.

Don’t forget to leave your comments below.