Minimizing SME Bias of Subject Matter Experts through effective Project Management

“Bias and prejudice are attitudes to be kept in hand, not attitudes to be avoided.” Charles Curtis

The use of SMEs (Subject Matter Experts) is commonplace throughout the lifecycle of a project. The “experts”, as we generally refer to them, are typically functional experts in their respective roles that the project manager relies on for making delivery estimations and identifying potential risks to a project. Depending on the dynamics of your particular organization, such experts may come from the same functional team that will be responsible for the execution of the tasks for which the experts are providing input. On the other hand, they may be in a specialist division that deals with project set-up. In either case, there are a several risks which the project manager needs to look out for to avoid being given an impossible or very difficult delivery task.

In many business environments, the experts available for a project are often within the same functional group that will be responsible for the execution of tasks on a project. Let’s consider for example, Project Alpha. Project Alpha’s objectives include designing a new car headlamp for Automotive Company XYZ. The experts used for this project are likely to include engineers, managers and/or senior personnel from various departments that will be responsible for the development of the product. The potential pitfall in using experts from the same group that will ‘implement it’ is that they accept the estimates without accounting for any bias that may be lurking in the estimates. This bias may be consciously or subconsciously included by the experts. Once the scope and requirements have been finalized for the project, the duration estimates are generally finalized with the input of the experts. In order to meet or exceed the duration estimates provided, the expert’s estimates may actually be “worst case” scenario unbeknown to the project manager, which has ensuing implications on project budget and schedule. The “Peter Principle” risks coming into play, by which the work expands to fill the time available and we do not get the optimum delivery result. Conversely, estimates from experts which are overly optimistic risk putting the delivery team to the sword – and put the project budget at risk. The conclusion: project managers should make sure that expert guidance is “peer reviewed” by an external person to ensure it is reasonable.

Bias is not isolated to the work estimates of a project. It is also connected to the level of risk involved, whether for an estimate of work or generally assessing project risks. Risk, according to PMBOK, is calculated as Probability * Impact. Yet risk depends on the project team’s appetite for it – particularly the client, Are they risk averse or risk-seeking? Depending on the environmental factors of your organization, how you obtain the probability and impact may vary. However, most calculations of impact will be based on the subjectivity and experience of experts. Take the example below, where the project team has agreed that impacts should fall into one of five categories (see figure 1.1 below), and any risk rated over a 2.0 impact should require mitigation planning.

Assessed Impact	Assigned Impact Value
Less than 5% variance in Schedule/Budget	1
5.1-10% variance in Schedule/Budget	3
10.1-15% variance in Schedule/Budget	5
15.1-20% variance in Schedule/Budget	7
Greater than 20% variance in Schedule/Budget	10

Figure 1.1

Which category a risk should fall into requires a degree of subjectively from those assigning the rating; which in most situations involves expert opinion. If the project manager relies only on a few experts to assess impact, potential identified risks may be understated, remain un-mitigated (not acted upon) and later cause issues for the project. Equally, true mitigation factors may not be properly assessed (the experts will not know as much about the project’s situation as other members of the team, or other stakeholders).

What tasks or steps should a project manager take to minimize expert bias? The project manager should, where possible, draw on a pool of experts, who are not on the project team nor responsible for the project’s tasks, to validate the assumptions, estimates, and risks provided on a project. Using a pool of experts should be considered more carefully, especially if the project will employ contractors. If a pool of experts is not available, the project manager should use assumptions and constraints methods in calculating estimates to be included in the response, as well as having PERT three point estimation [(O+(MLx4)+P)/6 ].

In the calculation of risk, the project manager should not limit the input to that of experts. Building on the risk example above, the probability of risk B was calculated using Monte Carlo analysis to be 47%. What value should be assigned to the “Impact”? If the project manager accepts only limited expert input, there are added risks that the impact may not be accurate, thus not mitigated.

Risk	Probability	Impact	Risk Rating
Risk B	0.47	?	?

Figure 1.2

This article suggests that each project risk should be carefully assessed by the stakeholder group for the project. The ‘aggregated mean value’ of the assigned impact should be used to make a reasoned decision. Whether or not you weigh each of the stakeholder’s responses the same will be based on your organizational factors. If you do weight their responses, think carefully about whether the weighting should or should not be shared with stakeholders, as this can lead to arguments or disagreements. People may not like to know that their option is weighted less than another’s, but sometimes it may be necessary. Below is an example.

Stakeholder	Stakeholder Group	Weighting	Impact Provided	Impact Score
Ann Smith	Expert	1	5	5
Bill Jones	Customer	0.5	7	3.5
Chris White	Sponsor	0.75	10	7.5
			Impact score Aggregate Mean	5.33

Figure 1.3

If we go back and use the aggregate impact, we find the risk rating as follows:

Risk	Probability	Impact	Risk Rating
Risk B	0.47	5.33	2.5051

Figure 1.4

Summary
Using Aggregate Risk Rating, and putting in place “Expert Pools” are just a few examples of how project managers can take steps to minimize the bias of expert opinion in making decisions for their project (throughout the various phases), to make sure the overall project risk and estimates of work are reasonable for the tasks required. Using a simple risk table (Probability x Impact) to validate your information can help you achieve this in a measured way.

Don’t forget to leave your comments below

---

Gary Hamiltonis the Manager of the PMO and Governance within Bank of America’s Learning and Leadership Development Products organization. Gary has 14 years of project and program management experience in IT, Finance and HR. He holds an advanced MBA degree in Finance and several certifications and credentials program management including PMI’s PgMP® (and PMP®.. Gary can be contacted through LinkedIn.

Gareth Byatt is Head of the IT Global Program Management Office for Lend Lease Corporation. He’s a PgMP®, PMP® and PRINCE2 practitioner, and holds an MBA and first-class undergraduate management degree. Gareth has 13 years of project and program management experience in IT and construction. He can be contacted through LinkedIn.

Jeff Hodgkinsonis the IT Cloud Program Manager for Intel Corporation. He is a 30-year veteran of Intel with a progressive career as a Program/Project Manager. Jeff’s credentials include PMI’s PgMP® and PMI-RMP® (Risk Management Professional) Jeff was 2nd place finalist for the 2009 Kerzner International Project Manager of the Year Award. He can be reached through LinkedIn.