Why is Benefits Management so Hard to do?

Almost every company accepts the need to manage project benefits, but very few actually do it!

Earlier this year, I started a discussion and poll on the topic of why benefits management is so hard to do. The response was intense, with over eighty-two PMO managers, consultants and professionals related to the topic. This article summarizes and comments on these responses and draws some conclusions.

The poll asked respondents to select one of five causes why benefits management is so hard to do. The results of the poll follow. This article will be structured around these five potential causes and, of course, as one respondent suggested, all of the above may be the winner.

No defined governance around benefits

Not surprisingly, this was the clear winner, as these are PMO-related professionals and the second word PMO people say when we wake up in the morning is probably governance (I will leave the first to your imagination). Governance structures for portfolios, programs and projects in most organizations have been designed to execute projects and deliver outputs, not deliver benefits or business results. No different than other areas, governance for benefits management should include guiding principles, definitions, roles and responsibilities, processes, templates and tools.

Let’s see what we got from the discussion.

As a starting point, Benefits Realization Management needs a business case on its own, as it requires resources. As Bruce Bozza, PMO/PM at Fujitzu in the UK states, “BR is a management process … it takes resources to effectively implement. It is not a magic wand. Its asserted value will challenge many senior managers’ mindsets. It can only be optimally implemented with appropriate sponsorship (top down) and associated governance.” Bruce touches on two key areas: resources and attention. Resources will be needed, probably dedicated, to implement BR. Second: attention from key executives and managers to participate in the governance bodies required to make BR work. So there is a cost — what about the benefits? If we consider that 20 to 40% of investment in IT is wasted every year (Gartner, IBM), decide where your organization sits compared to the norm, and then do the math with your yearly portfolio budget, and there is usually enough room to give BR a serious try.

“Cost/benefit do change depending on the market conditions, but some managers don’t want to do this analysis because of fear. If their pet project fails the analysis test, their position is at stake,” says Ravi Shankar, Senior Consultant at DWS, Melbourne, exploring the governance topic of periodic reviews of business case, as conditions usually change. In addition to fear, I would add that updating a business case because of market conditions is seldom a priority because there is no trigger. The only way to avoid this is periodic reviews, in addition to stage gates at approval, delivery, etc. The only exception to this is changes, which do have a trigger. The problem, as Linda Watts, PM at Coop Bank in the UK states, “The impact of changes to a project are not always managed against the impact of this against the benefits.” In my opinion, not always is very generous, as it is very seldom that changes are assessed against the business case. Of course, that only material changes are relevant, but not assessing changes against the business case misses the opportunity to ask an excellent question: when a change does not impact benefits, why are we doing it? Of course, there are always good reasons, but it is a good question nevertheless.

One of the main barriers regarding benefits realization management, particularly in IT, is that business resources — those that realize benefits, are not usually under the influence of a PMO. On this topic, Gillian Milne, Head of the PMO/CMO at Rand Merchant Bank in Johannesburg, South Africa, says, “However the PMO is often best placed to track what has actually been done against what was expected. As soon as benefit tracking is embedded more benefits are realized due to the focus, and business cases become more robust.” As Gillian puts it, a mature PMO is likely the best organizational unit to take the challenge of benefits realization, which cuts across organizational barriers, as long as there is a strong mandate and clear governance supporting this effort at the enterprise level. Success in this area can naturally position a PMO to evolve into an SMO or Strategy Management Office, focused on the execution of strategy, not just delivery of projects.

“Let’s be honest: PMOs have struggled with converting benefits realization from a concept into reality for a long time now,” says Calum Robertson, Principal Portfolio Analyst at Auckland Council, New Zealand. Calum is recognizing the need to do something different and provides concrete recommendations: “Change in culture from thinking about projects to thinking about investments. Projects are expected to be completed ‘on time, on budget’ but investments are expected to yield a return.” Calum hits on the focus on delivery that has driven most PMOs, with a narrow view limited to the triple constraints and with little or no attention to benefits. Strong governance and sponsorship is required to implement this cultural change. Another topic Calum hits is the need for accountabilities that transcend project closure, “Maintaining the project spotlight (and dashboard reporting) beyond project closure.” As soon as projects are delivered, they disappear from the radar screen, and benefits promised for upcoming years are very seldom considered when analyzing the impact of most recent projects. Again, this points to resources and tools to do it properly, but can we afford not to and remain flying blind on benefits?

Benefits are not standard or comparable

While this cause received 22% of the vote, there were very few comments in the discussion. In a way, it is related to governance, which should define standards for calculating benefits so they are comparable across investments. This requires a framework for benefits realization that includes not only the calculation of benefits, but definitions and rules that determine when a business case is needed and the rigor and detail required, depending on the type of investment and freedom to invest. For example, an infrastructure project required to replace obsolete technology or a legislated change have little or no freedom to invest. On the other extreme, a venture usually has high freedom to invest and requires more scrutiny and higher returns.

Fiona Binney, RM at Intergen in New Zeeland, expands the discussion into the topic of tangible VS non-tangible: “First they need to be identified as tangible or non-tangible, the tangible ones then need to be baselined as mentioned above, and they also need to be assigned to a benefits manager who will be responsible for reporting on the delivery of these benefits for a dedicated length of time.” It is clear that tangible benefits should be the ones subject to measurement, but non-tangibles, like customer satisfaction, employee morale, etc., can still be assessed.

Joe Lunn, Portfolio Manager at Zurich North America in Chicago, expands the categories for benefits: “We segment benefits into 1) hard, 2) quantifiable soft and 3) qualitative categories and provide sponsors and project teams with definitions and monitor submissions to assure accurate use of these definitions.” I want to direct your attention to the term “definitions” as they are integral to make this approach successful, and again, strong governance is required.

Even within the same type of investment, the estimate of benefits should be comparable. As a simple example, if two projects are projecting savings on production costs for product A, as a minimum they should use the same production forecast for product A, and you would be surprised at how often this is not the case. Referring estimates of benefits to units of output is always a best practice, as projects either increase these units of output (increase revenue) or reduce variable cost per unit. Only fixed costs are different but usually easier to verify. Once this is done, the next step is to define standard formulas that make all the assumptions explicit. As an example, for headcount reduction you need to specify not how many people you will reduce, but the change you are expecting in units of output per person, what the current units or baseline is and the average loaded cost per person. This way, your estimate does not depend on whether sales go up or down, or whether salaries go up — there is no place to hide.

Jodelen Mitra, PM practitioner from the Philippines, summarizes this topic very well: “If the logical framework has been well-defined from the beginning and the performance plan where indicators and targets developed, then benefits management will be easier.”

Accountability for benefits is misplaced

This cause came third at 19.5% of responses, very close to the second one, and is at the core of this discussion as the issue of accountability for benefits lies on the frontier between the service provider, likely an IT department, and the consumer of the output of the project, likely a business unit. When a business case for a project is created, it usually involves the participation of business resources, which provide the projections and assumptions regarding benefits. At the end, it is the business case for the project, and the project manager may end up accountable for delivering benefits on which s/he has no control.

On the topic of accountability, Ben Reardon, Senior Project Officer at the University of Western Sydney, Australia, says, “So often as project professionals, we focus on a variation of ‘on time, on scope/quality, on cost’ that we forget that the measure of what a project is for is how (and how much) it delivers benefits.” Here, Ben is opening a discussion on the two dimensions of accountability for benefits: one is to measure and the other is to realize the benefit. The former requires skill to define the metrics and collect measurement, and the delivery organization is likely better prepared to do this than the business. For the benefit itself, it is clearly the business that is accountable, the sponsor, manager or executive who pitched the project.

Emphasizing the need to see projects as investments, Steve Prothero, Principal at True to You, Camberra, Australia, states, “It is a combination of lack of ownership (i.e., it is not my ACTUAL money) so there is a level of abstraction that is compounded by lack of knowledge of IT.” And Bruze Bozza complements this discussion with a suggestion to make money real: “Enforcing accountability and linking rewards will help generate the business culture change required at senior levels.” Linking rewards to achievement of business benefits is probably at the top of the maturity scale for benefits realization management. The problem is to get there without this link, as money in a business case is not real, many projects can claim success over the same benefits, as there is no way to reconcile what all the different projects are promising. In a consulting engagement a few years ago, I reconciled all the promises for productivity improvement that had been made, and they represented savings sufficient to reduce the workforce by 45% and, of course, not a single person had been reduced. Because business cases are done in isolation, there is no ceiling for what can be promised, outside of sanity, that is.

To round up the discussion on accountability, a framework for benefits realization management needs to assign accountability at three levels:

• Project Managers are accountable for delivering capabilities
• Program Managers and Business Managers can be accountable for delivering business outcomes
• Executives are accountable for the link between business outcomes and financial results, because this is, in essence, the strategy of the business, which should consider all the external forces.

Under this framework, the frontier for accountability resides with the business outcomes (i.e., number of repeat customers increased, hours of truck utilization increased, traffic through Internet site increased, etc.). It is the executives who, in their strategy, make the connection between these outcomes and the financial results of increased revenue, reduced cost, etc.; only they can be accountable for this. The rest of the organization is accountable for delivering to these business outcomes, which also happen to be the usual performance metrics the organization already tacks. This way, benefits realization management can be done in a way that assigns accountability where it belongs and leverages existing metrics.
Unrealistic business cases

This cause came as a distant fourth with 14.6%, but this could be related to the fact that governance, accountability and business cases not comparable are all related to unrealistic business cases. In other words, with strong governance and clear accountability, people would be more careful when making promises for benefits. If the checks and balances and process are not in place, business case money is free!

As Linda Watts states: “Firstly the benefits that are identified at the start of a programme are often hypothetical figures stated by an Exec team who are not close enough to the detail to be able to validate. This leads to a lack of buy-in from individual teams who are left to deliver them.” In the executive suite, estimates for benefits tend to be high level and generous, particularly when it comes to pet projects. Then these estimates of benefits are passed to a director or manager in the business who rarely buys in or feels accountable for those numbers.

On the topic of accuracy, business cases tend to be completely disparate. On one hand, the estimates of investment are usually detailed because this is something we do well and have information for, so we take it to a level of accuracy that does not correspond with the estimate of benefits against which it will be compared. Estimates of benefits are, by nature, nothing more than an educated guess. Joe Lunn brings the idea of stage gating to the discussion: “One way to assure greater accuracy of benefits targeting is to revisit the benefit estimate originally presented during project approval at least two more points in the project lifecycle. First is after requirements are completed. Reassess whether the original benefits can be achieved given these requirements. Second, assure that contractors or vendors performing development work understand the targeted benefits and indicate comfort that their development work will contribute to achieving these.” When stage gates for the business case are defined in advance, sponsors tend to be more careful with the initial estimates, as they know they will have to defend them at later stages. When this is not the case, they will try to make the business case as attractive as possible to get the project approved.

Benefits are not verifiable

This cause came last with only 7.3% of responses. However, most of the discussion had to do with this topic, which, in my opinion, is at the core of the inability of organizations to implement benefits realization. Let’s look at the discussion:
Kik Piney, Professional Training & Coaching Consultant and Contractor in France, brings the complexities of causality into the discussion: “Benefits are badly managed because there is no agreed way to manage them well — because there is no effective mechanism for measuring progress towards achieving them. This statement needs a bit of explanation. As a starting position, it is useful to clarify some PMI concepts: PMI talks about ‘benefits realization’ in the Standard for Program Management. However, what a program provides, at best is ‘benefits enablement’ — i.e., a change to the operational environment which should contribute to improving business success. So what does this mean? Consider the central benefits planning analysis tool: the ‘outcome map’ that is designed to map from deliverables to outcomes, and from outcomes to benefits. This can be envisaged as an ‘if-tree’ in an ‘assumptions forest’:

• if we can create the (project-related) deliverables as defined,
• and if they can be used to generate the outcomes in the outcomes map
• and if the outcomes would enable the realization of the forecast benefits, then,
• assuming the business environment is still as expected in the initial analysis, we should be able to achieve the predicted result. This means that, until it is too late, there are no reliable means of tracking progress towards success.”

The need to understand and manage the relationships between projects, outputs and outcomes, as stated by Kik, clearly explains why the traditional approach of ranking business cases only makes sense in an operational scenario, where there is a direct cause and effect relationship between projects and some financial return, like reduce a certain cost. In today’s world, most organizations are in a transformational/adaptation scenario, where capabilities (usually output of projects) have to integrate with others before generating business outcomes.

Kik also mentions the forest of assumptions, and these imply risk. We are very good at estimating delivery risk, but what about benefit risk? When the causality relationships are compounded with delivery and benefit risks, it is easy to understand why organizations shy away from benefits realization management, as it looks too complicated, and traditional portfolio management applications do not tackle these issues.

Bruce Bozza complements the discussion: “Sound BR does not just involve defining the end benefits and measures … critically it requires understanding and measuring the lead indicators to outcome/benefit generation to enable corrective decision making in a timely manner. Similarly, it requires an understanding of the component relationships and interdependencies … e.g., what if I stop A, what if we speed up B, where to cut funding in bad times, and so on.” The concept of outcomes as leading indicators brings the whole topic of benefits realization closer, as programs can be defined around business outcomes that represent leading indicators. For example, a program could be defined to reduce the cycle time for preparing a proposal, a business outcome that can act as a leading indicator for another business outcome: percent of proposals won, a lagging indicator and definitely a good predictor of revenue increased, a financial outcome. Once again, this reinforces the need to understand causality relationships.

The achievement of financial outcomes, even business outcomes, usually depends on many streams of causality, which make the estimation and the verification of benefits impossible. Gregory Ellis, PM at Dell, Japan, says, “It’s all the talk these days about how PMOs can no longer satisfy themselves with delivering specific business requirements on scope/budget/schedule, but have to take ownership on the business outcomes, blah blah blah. Baloney. This notion fails because of the same reason that attempts to apply Six Sigma to project management fails: there are too many uncertainties and variables at play to offer concrete visibility and control. Did the Sales department’s revenue increase 20% last quarter because you deployed the Sales Force Super-Duper Empowero-matic, or because of the staff augmentation and new Center of Excellence training program that HR implemented, or because of the flex-time, or because of Product Development’s unveiling of a popular new wammy-jammy that captured market share? Ugh … maybe all of them? None? It’s a pipe dream that the Board has thought up because it wants concrete answers on whether its investments are providing raw, bankable value, and a dream that is not rational.”

One more element in this discussion is the topic of baselining. Kiron Bondale, Project Portfolio & Change Management Practitioner in Toronto, Canada, states: “Oftentimes, no baseline exists against which to compare the benefits. In such cases, the project should be responsible for identifying the appropriate metric, gathering a baseline and then delivering benefits against that baseline.” Everything in benefits realization management is incremental, cash in and cash out should be the foundations of a business case, and if there is no baseline, verification of benefits cannot be done.

Conclusion

The compilation of the different points of view regarding why benefits management is so difficult to do seems to converge in two main conclusions:

• Strong governance with clear accountabilities and a defined framework are needed, and this has a cost that is not trivial
• There is an intrinsic problem in measuring and managing benefits due to causality relationships that makes it difficult, if not impossible, to do

The first conclusion was easy to anticipate. The second points to the need for innovation and to do things in a different way. There is basically one way of doing portfolio management, and it doesn’t work most of the time. There is a parallel to the use of waterfall in the past for every project. It worked for some of them, but it never worked for those projects with high volatility and uncertainty, in which creating a schedule or trying to manage from it was an exercise of futility. In a similar way, there is a need to do things in a different way in benefits realization, because the current approach, which was developed for a more stable environment where most projects were operational in nature, is no longer the norm but the exception. Throwing governance on top of an approach that doesn’t work is not going to make things better; on the contrary, it will highlight the shortcomings and put the whole concept of benefits realization management at risk.

The main points from this discussion have also been used to produce a short video called “Benefits realization, the problem to be solved”: This video also includes an alternative approach: top-down portfolio management, which I have proposed in other articles, in which benefits are estimated at the portfolio level and as a result of the implementation of a strategy or plan. These benefits are then allocated based on the relative contribution of each initiative to the intermediate and final results. In this approach, benefits realization is focused on the achievement of business outcomes, which are more tangible and easier to measure, making benefits realization more effective and easier to implement.

To summarize, the area of benefits realization management is an accepted concept in most organizations, and sometimes even a mandate for heads of a PMO. However, there is a gap between intention and execution. Throwing governance behind this could make things even worse. The reality is that until the intrinsic limitations introduced by causality relationships are addressed, this endeavour will neither be realized nor beneficial.

Don’t forget to leave your comments below.