Stop Promising Miracles: Wideband Delphi Team Estimation, Part 2
In the first article in this three-part series I described some of the reasons why estimates often deviate wildly from reality, and I introduced an effective technique for group estimation called Wideband Delphi (see Figure 1). This article continues the description of Wideband Delphi, showing how the individual preparation and estimation meeting steps are carried out.
Let’s assume you wish to estimate the total amount of work effort (typically expressed in labor hours) needed to complete a certain project. The estimation process begins with each participant independently developing an initial list of the tasks that will have to be completed, using a form like that in Figure 2. Figure 2 illustrates a very small estimation activity, developing the requirements for a new project cleverly titled Project X. The person who completed this form initially identified five tasks that would be involved in developing requirements, the first five items listed under the “Task” column heading.
Each participant then estimates the effort each task will consume. Decompose each activity down into tasks that are small enough to estimate accurately. State the tasks clearly because someone will have to merge all of the participant task lists into a single composite list. Total the estimates you produce for each project task, in the agreed-upon units, to generate your initial overall estimate. Returning to Figure 2, the column headed “Initial Estimate” shows the labor hours this estimator assigned to each of the five identified tasks, for a total of 67 hours estimated for developing the requirements for this project.
In addition to identifying the project tasks, separately record any tasks for related or supporting activities, such as quality tasks, process activities, and waiting time that impose delays. In my first Wideband Delphi session, every participant forgot to list tasks dealing with quality control and assurance, configuration management, and process-related activities on the first cycle. We caught this quickly and added them in for the next iteration. Be sure to include rework tasks following testing or inspection activities. Reworking to correct defects is a fact of life so you should plan for it. If you’re estimating a project schedule, also identify any overhead activities that aren’t specific to the project that you might have to build into your planning. These include meetings, vacation, training, other project assignments, and myriad other things that suck time out of your day.
Since radically different assumptions can lead to wide estimate variations, record any assumptions you made while preparing your estimates. For example, if you assumed that you will purchase a specific component library or reuse one from a previous project, write that down. Another estimator might assume that the project will custom-develop that component library, which will lead to a mismatch between your two overall estimates.
Keep the following estimation guidelines in mind, even if they don’t reflect exactly how the project work will be performed:
- Assume one person will perform all tasks.
- Assume all tasks will be performed sequentially; don’t worry about sequencing and predecessor tasks at this time.
- Assume the worker can devote uninterrupted effort to each task. This may seem absurdly optimistic but it simplifies the estimation process.
- In units of calendar time, list any known waiting times you expect to encounter between tasks. This will help you translate effort estimates into schedule estimates later on.
The moderator begins the estimation meeting by collecting the participants’ individual estimates and creating a chart such as the one shown in Figure 3. Each participant’s total project estimate is plotted as an X on the “Round 1” line. Each estimator can see where his initial value fits along the spectrum. The initial estimates probably will cover a frighteningly large range. Just imagine the different conclusions you might have collected had you asked just one of these five individuals for his estimate!
The moderator does not identify who created each estimate. This anonymity is an important aspect of the Delphi technique. Anonymity prevents an outspoken colleague from intimidating the other participants into seeing things his way. It also means team members are less likely to defer to a more respected participant’s judgment when their own analyses lead to different conclusions.
Each estimator reads his initial task list aloud, identifying any assumptions made and raising any questions or issues, without revealing their task estimates. Different participants will think of different tasks. Combining these individual task lists leads to a more complete list than any single estimator would produce. This approach will work for up to several dozen individual tasks. If you have more tasks than that you might want to break the problem into several subproblems and estimate them individually.
During this discussion, the team members also talk about their assumptions, estimation issues, and questions they have about the problem. As a result, the team will begin to converge on a shared set of assumptions and a common task list. Retain this final task list to use as a starting point the next time you must estimate a similar activity.
After this initial discussion, all participants modify their estimates concurrently (and silently) in the meeting room. They might revise their task lists based on the information shared during the discussion, and they’ll adjust individual task estimates based on their new understanding of the task scope or changed assumptions. All estimators can add new tasks to their forms and note any changes they wish to make to their initial task estimates. The net change for all tasks equals the change in that participant’s overall project estimate.
Let’s continue with the example in Figure 2. Perhaps this estimator learned that some of the user representatives were already identified so he could remove one hour from his estimate for that task. This change of minus 1 hour appears in the column headed “Change #1”. He might have concluded that additional user requirements workshops would be needed and that more reviewers should look at the resulting specification. This resulted in increased estimates for those two tasks, as well as an increase in the estimated time to write the requirements specification. Maybe a developer in the workshop stated that he expected to use a more efficient prototyping tool, so our estimator reduced that task effort by eight hours. However, during the discussion for estimation round 1, someone else’s task list included the step of correcting errors found during the requirements review. Our estimator added that new task to his list, estimating five hours of effort. The cumulative change from these adjustments made during the second estimation round was an increase of 10 hours, for a revised total estimate of 78 hours.
The moderator collects the revised overall estimates and plots them on the same chart, on the Round 2 line. As Figure 4 illustrates, the second round might lead to a narrower distribution of estimates centered around a higher mean than the mean of the Round 1 values. Additional rounds should further narrow the distribution. The cycle of revising the task list, discussing issues and assumptions and preparing new estimates continues until one of the following conditions is fulfilled:
- You have completed four estimation rounds.
- The estimates have converged to an acceptably narrow range (defined in advance).
- The allotted estimation meeting time (typically two hours) is over.
- All participants are unwilling to alter their latest estimates.
The moderator keeps the group on track, time-boxing discussions for each round to 15 or 20 minutes to avoid endless rambling. While preserving the anonymity of individual estimates is important for the first couple of rounds, the team members might agree at some point to put all their cards on the table and reach closure through an open discussion. This gives them a chance to discuss tasks for which their estimates vary substantially. Otherwise, though, the moderator should not identify the individual who produced each final estimate until the session is completed.
The final article in this series will describe the wrap-up process steps of assembling tasks into a single comprehensive list, reviewing the results, and completing the Wideband Delphi estimation session.
Don’t forget to leave your comments below.