Stop Promising Miracles: Wideband Delphi Team Estimation, Part 1
When I was a software developer I once received a phone call from my manager. “Got an hour?” he asked. My manager had received what seemed to be a trivial request from one of our customers. However, by the time I had understood the customer’s real need and wrote a small application to satisfy it, I had devoted 100 hours to this project. Not all project estimates are off by a factor of 100 but this is a striking example that it is indeed possible to go that far wrong.
This three-part series (adapted from my book Practical Project Initiation: A Handbook with Tools) describes an effective technique for group estimation called Wideband Delphi. The Wideband Delphi method pools the experience and knowledge of several estimators to come up with more realistic estimates than any one of them could have devised his own.
Most software professionals must provide estimates for their work, but few of us are skillful estimators. Many of us haven’t been trained in estimation techniques. We’re too optimistic, with short memories that mask the painful overruns from previous projects. And we often overlook necessary aspects of an activity, so when we eventually confront those tasks, we either perform them—thereby exceeding our estimates—or skip them, perhaps compromising quality in the process.
The “Cone of Uncertainty” illustrated in Figure 1 cautions that estimates created very early in a project have very large deviations from the ultimate reality (see Steve McConnell, Software Estimation, Microsoft Press, 2006). Preliminary estimates based on limited information and many assumptions can be off by a factor of four on both the high side and the low side. Nonetheless, stakeholders do want some kind of estimates early in the project, recognizing that estimates will become both more accurate and more precise as the project continues.
There are several ways to become a better estimator. The most basic is to record effort, duration, or size estimates along with your estimating processes and assumptions, and then record the actual results from each estimated activity. Comparing actual outcomes to the estimates helps you to generate more accurate estimates in the future. Estimation procedures and templates that itemize tasks help avoid the common problem of overlooking necessary work.
Another approach builds on the principle that multiple heads are better than one. Developed at the Rand Corporation in 1948, the Delphi estimation method uses a small team of experts to anonymously generate individual estimates from a problem description and reach consensus on a final set of estimates through iteration. In the early 1970s, Barry Boehm and his Rand colleagues modified this method into Wideband Delphi, which included more estimation team interaction. Mary Sakry and Neil Potter of The Process Group (www.processgroup.com) later created a repeatable procedure for performing Wideband Delphi estimation on software projects.
Using the Wideband Delphi method provides several advantages over obtaining just one estimate from a single individual. First, it helps build a comprehensive task list or work breakdown structure for major activities, because each participant will think of different tasks to perform. The consensus approach helps eliminate bias in estimates produced by self-proclaimed experts, inexperienced estimators, or influential individuals who have hidden agendas or divergent objectives. People are generally more committed to estimates they help produce than to those that others generate. No participant in an estimation activity knows the “right” answer; creating multiple estimates acknowledges this uncertainty. Finally, users of the Delphi approach exploit the value of iteration on any complex activity.
Wideband Delphi can be used to estimate virtually anything: the labor months needed to implement a specific subsystem, the lines of code or number of classes in an entire product, or the gallons of paint needed to redecorate your house. I used Wideband Delphi once with a process improvement group to estimate the effort it would take a particular organization to achieve a specific process improvement objective.
The Delphi method helps you develop a detailed work breakdown structure, which provides the foundation for bottom-up size, effort, or schedule estimation. The starting point for a Delphi session could be a specification of the problem being estimated, an initial high-level task list, or a preliminary project schedule. The outputs are: a detailed project task list; a list of associated quality, process-related and overhead tasks; estimation assumptions; and a set of task and overall project estimates, one from each participant.
Figure 2 illustrates the Wideband Delphi process flow. During planning, the project manager defines the problem being estimated and selects the participants. The kickoff meeting gets all estimators focused on the problem. Each participant then individually prepares his initial task lists and estimates. Estimators bring these items to the estimation meeting, during which several estimating cycles lead to a more comprehensive task list and a revised set of estimates. The moderator or project manager then consolidates the assorted estimation information offline, and the team reviews the estimation results. The session is completed when some predetermined exit criteria are satisfied. The resulting range of estimates will almost certainly be a more realistic predictor of the future than any single estimate could be. Let’s look at each of these process steps in turn.
A Wideband Delphi session begins with defining and scoping the problem. Large problems are broken down into manageable portions that can be estimated more accurately, perhaps by different teams. The person who initiated the estimation activity assembles a problem specification that will give the participants enough information to produce informed, credible estimates.
The estimation participants include a moderator, who plans and coordinates the activity, the project manager, and two to four other estimators. The moderator should be informed enough to participate as an estimator, but the moderator acts as an impartial facilitator who won’t skew the results with his own biases or insights. The moderator and project manager select participants who understand the problem or project and the associated estimation issues.
An initial kickoff meeting gets all participants up to speed on the estimation problem. The moderator explains the Wideband Delphi method to team members who aren’t familiar with it. He supplies the other estimators with the problem specification and any assumptions or project constraints.
The team reviews the estimation objectives and discusses the problem and any estimation issues. The participants agree on the estimation units they will use, such as weeks, labor hours, dollars, or lines of code. If the moderator concludes that all team members are sufficiently knowledgeable to contribute to the estimation activity, the group is ready to roll. The moderator might elect to replace certain estimators with others who can generate more accurate estimates.
To determine whether you’re ready to proceed with the Wideband Delphi session, check your entry criteria, the prerequisites that must be satisfied for you to proceed with subsequent process steps. Before you dive into the estimation exercise, ensure that the following conditions are satisfied:
- Appropriate team members have been selected.
- The kickoff meeting has been held.
- The participants have agreed on the estimation goal and units.
- The project manager can participate in the session.
- The estimators have the information they need to participate effectively.
The next article in this series will describe what the estimation participants do during their individual preparation and also how the all-important estimation meeting is carried out.
Don’t forget to leave your comments below.