Author: Fernando Santiago

Why is Benefits Management so Hard to do?

Written by Fernando Santiago on June 5, 2012. Posted in Articles. 2 Comments on Why is Benefits Management so Hard to do?

Almost every company accepts the need to manage project benefits, but very few actually do it!

Earlier this year, I started a discussion and poll on the topic of why benefits management is so hard to do. The response was intense, with over eighty-two PMO managers, consultants and professionals related to the topic. This article summarizes and comments on these responses and draws some conclusions.

The poll asked respondents to select one of five causes why benefits management is so hard to do. The results of the poll follow. This article will be structured around these five potential causes and, of course, as one respondent suggested, all of the above may be the winner.

fernandjune61

No defined governance around benefits

Not surprisingly, this was the clear winner, as these are PMO-related professionals and the second word PMO people say when we wake up in the morning is probably governance (I will leave the first to your imagination). Governance structures for portfolios, programs and projects in most organizations have been designed to execute projects and deliver outputs, not deliver benefits or business results. No different than other areas, governance for benefits management should include guiding principles, definitions, roles and responsibilities, processes, templates and tools.

Let’s see what we got from the discussion.

As a starting point, Benefits Realization Management needs a business case on its own, as it requires resources. As Bruce Bozza, PMO/PM at Fujitzu in the UK states, “BR is a management process … it takes resources to effectively implement. It is not a magic wand. Its asserted value will challenge many senior managers’ mindsets. It can only be optimally implemented with appropriate sponsorship (top down) and associated governance.” Bruce touches on two key areas: resources and attention. Resources will be needed, probably dedicated, to implement BR. Second: attention from key executives and managers to participate in the governance bodies required to make BR work. So there is a cost — what about the benefits? If we consider that 20 to 40% of investment in IT is wasted every year (Gartner, IBM), decide where your organization sits compared to the norm, and then do the math with your yearly portfolio budget, and there is usually enough room to give BR a serious try.

“Cost/benefit do change depending on the market conditions, but some managers don’t want to do this analysis because of fear. If their pet project fails the analysis test, their position is at stake,” says Ravi Shankar, Senior Consultant at DWS, Melbourne, exploring the governance topic of periodic reviews of business case, as conditions usually change. In addition to fear, I would add that updating a business case because of market conditions is seldom a priority because there is no trigger. The only way to avoid this is periodic reviews, in addition to stage gates at approval, delivery, etc. The only exception to this is changes, which do have a trigger. The problem, as Linda Watts, PM at Coop Bank in the UK states, “The impact of changes to a project are not always managed against the impact of this against the benefits.” In my opinion, not always is very generous, as it is very seldom that changes are assessed against the business case. Of course, that only material changes are relevant, but not assessing changes against the business case misses the opportunity to ask an excellent question: when a change does not impact benefits, why are we doing it? Of course, there are always good reasons, but it is a good question nevertheless.

One of the main barriers regarding benefits realization management, particularly in IT, is that business resources — those that realize benefits, are not usually under the influence of a PMO. On this topic, Gillian Milne, Head of the PMO/CMO at Rand Merchant Bank in Johannesburg, South Africa, says, “However the PMO is often best placed to track what has actually been done against what was expected. As soon as benefit tracking is embedded more benefits are realized due to the focus, and business cases become more robust.” As Gillian puts it, a mature PMO is likely the best organizational unit to take the challenge of benefits realization, which cuts across organizational barriers, as long as there is a strong mandate and clear governance supporting this effort at the enterprise level. Success in this area can naturally position a PMO to evolve into an SMO or Strategy Management Office, focused on the execution of strategy, not just delivery of projects.

“Let’s be honest: PMOs have struggled with converting benefits realization from a concept into reality for a long time now,” says Calum Robertson, Principal Portfolio Analyst at Auckland Council, New Zealand. Calum is recognizing the need to do something different and provides concrete recommendations: “Change in culture from thinking about projects to thinking about investments. Projects are expected to be completed ‘on time, on budget’ but investments are expected to yield a return.” Calum hits on the focus on delivery that has driven most PMOs, with a narrow view limited to the triple constraints and with little or no attention to benefits. Strong governance and sponsorship is required to implement this cultural change. Another topic Calum hits is the need for accountabilities that transcend project closure, “Maintaining the project spotlight (and dashboard reporting) beyond project closure.” As soon as projects are delivered, they disappear from the radar screen, and benefits promised for upcoming years are very seldom considered when analyzing the impact of most recent projects. Again, this points to resources and tools to do it properly, but can we afford not to and remain flying blind on benefits?

Benefits are not standard or comparable

While this cause received 22% of the vote, there were very few comments in the discussion. In a way, it is related to governance, which should define standards for calculating benefits so they are comparable across investments. This requires a framework for benefits realization that includes not only the calculation of benefits, but definitions and rules that determine when a business case is needed and the rigor and detail required, depending on the type of investment and freedom to invest. For example, an infrastructure project required to replace obsolete technology or a legislated change have little or no freedom to invest. On the other extreme, a venture usually has high freedom to invest and requires more scrutiny and higher returns.

Fiona Binney, RM at Intergen in New Zeeland, expands the discussion into the topic of tangible VS non-tangible: “First they need to be identified as tangible or non-tangible, the tangible ones then need to be baselined as mentioned above, and they also need to be assigned to a benefits manager who will be responsible for reporting on the delivery of these benefits for a dedicated length of time.” It is clear that tangible benefits should be the ones subject to measurement, but non-tangibles, like customer satisfaction, employee morale, etc., can still be assessed.

Joe Lunn, Portfolio Manager at Zurich North America in Chicago, expands the categories for benefits: “We segment benefits into 1) hard, 2) quantifiable soft and 3) qualitative categories and provide sponsors and project teams with definitions and monitor submissions to assure accurate use of these definitions.” I want to direct your attention to the term “definitions” as they are integral to make this approach successful, and again, strong governance is required.

Even within the same type of investment, the estimate of benefits should be comparable. As a simple example, if two projects are projecting savings on production costs for product A, as a minimum they should use the same production forecast for product A, and you would be surprised at how often this is not the case. Referring estimates of benefits to units of output is always a best practice, as projects either increase these units of output (increase revenue) or reduce variable cost per unit. Only fixed costs are different but usually easier to verify. Once this is done, the next step is to define standard formulas that make all the assumptions explicit. As an example, for headcount reduction you need to specify not how many people you will reduce, but the change you are expecting in units of output per person, what the current units or baseline is and the average loaded cost per person. This way, your estimate does not depend on whether sales go up or down, or whether salaries go up — there is no place to hide.

Jodelen Mitra, PM practitioner from the Philippines, summarizes this topic very well: “If the logical framework has been well-defined from the beginning and the performance plan where indicators and targets developed, then benefits management will be easier.”

Accountability for benefits is misplaced

This cause came third at 19.5% of responses, very close to the second one, and is at the core of this discussion as the issue of accountability for benefits lies on the frontier between the service provider, likely an IT department, and the consumer of the output of the project, likely a business unit. When a business case for a project is created, it usually involves the participation of business resources, which provide the projections and assumptions regarding benefits. At the end, it is the business case for the project, and the project manager may end up accountable for delivering benefits on which s/he has no control.

On the topic of accountability, Ben Reardon, Senior Project Officer at the University of Western Sydney, Australia, says, “So often as project professionals, we focus on a variation of ‘on time, on scope/quality, on cost’ that we forget that the measure of what a project is for is how (and how much) it delivers benefits.” Here, Ben is opening a discussion on the two dimensions of accountability for benefits: one is to measure and the other is to realize the benefit. The former requires skill to define the metrics and collect measurement, and the delivery organization is likely better prepared to do this than the business. For the benefit itself, it is clearly the business that is accountable, the sponsor, manager or executive who pitched the project.

Emphasizing the need to see projects as investments, Steve Prothero, Principal at True to You, Camberra, Australia, states, “It is a combination of lack of ownership (i.e., it is not my ACTUAL money) so there is a level of abstraction that is compounded by lack of knowledge of IT.” And Bruze Bozza complements this discussion with a suggestion to make money real: “Enforcing accountability and linking rewards will help generate the business culture change required at senior levels.” Linking rewards to achievement of business benefits is probably at the top of the maturity scale for benefits realization management. The problem is to get there without this link, as money in a business case is not real, many projects can claim success over the same benefits, as there is no way to reconcile what all the different projects are promising. In a consulting engagement a few years ago, I reconciled all the promises for productivity improvement that had been made, and they represented savings sufficient to reduce the workforce by 45% and, of course, not a single person had been reduced. Because business cases are done in isolation, there is no ceiling for what can be promised, outside of sanity, that is.

To round up the discussion on accountability, a framework for benefits realization management needs to assign accountability at three levels:

Project Managers are accountable for delivering capabilities
Program Managers and Business Managers can be accountable for delivering business outcomes
Executives are accountable for the link between business outcomes and financial results, because this is, in essence, the strategy of the business, which should consider all the external forces.

Under this framework, the frontier for accountability resides with the business outcomes (i.e., number of repeat customers increased, hours of truck utilization increased, traffic through Internet site increased, etc.). It is the executives who, in their strategy, make the connection between these outcomes and the financial results of increased revenue, reduced cost, etc.; only they can be accountable for this. The rest of the organization is accountable for delivering to these business outcomes, which also happen to be the usual performance metrics the organization already tacks. This way, benefits realization management can be done in a way that assigns accountability where it belongs and leverages existing metrics.
Unrealistic business cases

This cause came as a distant fourth with 14.6%, but this could be related to the fact that governance, accountability and business cases not comparable are all related to unrealistic business cases. In other words, with strong governance and clear accountability, people would be more careful when making promises for benefits. If the checks and balances and process are not in place, business case money is free!

As Linda Watts states: “Firstly the benefits that are identified at the start of a programme are often hypothetical figures stated by an Exec team who are not close enough to the detail to be able to validate. This leads to a lack of buy-in from individual teams who are left to deliver them.” In the executive suite, estimates for benefits tend to be high level and generous, particularly when it comes to pet projects. Then these estimates of benefits are passed to a director or manager in the business who rarely buys in or feels accountable for those numbers.

On the topic of accuracy, business cases tend to be completely disparate. On one hand, the estimates of investment are usually detailed because this is something we do well and have information for, so we take it to a level of accuracy that does not correspond with the estimate of benefits against which it will be compared. Estimates of benefits are, by nature, nothing more than an educated guess. Joe Lunn brings the idea of stage gating to the discussion: “One way to assure greater accuracy of benefits targeting is to revisit the benefit estimate originally presented during project approval at least two more points in the project lifecycle. First is after requirements are completed. Reassess whether the original benefits can be achieved given these requirements. Second, assure that contractors or vendors performing development work understand the targeted benefits and indicate comfort that their development work will contribute to achieving these.” When stage gates for the business case are defined in advance, sponsors tend to be more careful with the initial estimates, as they know they will have to defend them at later stages. When this is not the case, they will try to make the business case as attractive as possible to get the project approved.

Benefits are not verifiable

This cause came last with only 7.3% of responses. However, most of the discussion had to do with this topic, which, in my opinion, is at the core of the inability of organizations to implement benefits realization. Let’s look at the discussion:
Kik Piney, Professional Training & Coaching Consultant and Contractor in France, brings the complexities of causality into the discussion: “Benefits are badly managed because there is no agreed way to manage them well — because there is no effective mechanism for measuring progress towards achieving them. This statement needs a bit of explanation. As a starting position, it is useful to clarify some PMI concepts: PMI talks about ‘benefits realization’ in the Standard for Program Management. However, what a program provides, at best is ‘benefits enablement’ — i.e., a change to the operational environment which should contribute to improving business success. So what does this mean? Consider the central benefits planning analysis tool: the ‘outcome map’ that is designed to map from deliverables to outcomes, and from outcomes to benefits. This can be envisaged as an ‘if-tree’ in an ‘assumptions forest’:

if we can create the (project-related) deliverables as defined,
and if they can be used to generate the outcomes in the outcomes map
and if the outcomes would enable the realization of the forecast benefits, then,
assuming the business environment is still as expected in the initial analysis, we should be able to achieve the predicted result. This means that, until it is too late, there are no reliable means of tracking progress towards success.”

The need to understand and manage the relationships between projects, outputs and outcomes, as stated by Kik, clearly explains why the traditional approach of ranking business cases only makes sense in an operational scenario, where there is a direct cause and effect relationship between projects and some financial return, like reduce a certain cost. In today’s world, most organizations are in a transformational/adaptation scenario, where capabilities (usually output of projects) have to integrate with others before generating business outcomes.

Kik also mentions the forest of assumptions, and these imply risk. We are very good at estimating delivery risk, but what about benefit risk? When the causality relationships are compounded with delivery and benefit risks, it is easy to understand why organizations shy away from benefits realization management, as it looks too complicated, and traditional portfolio management applications do not tackle these issues.

Bruce Bozza complements the discussion: “Sound BR does not just involve defining the end benefits and measures … critically it requires understanding and measuring the lead indicators to outcome/benefit generation to enable corrective decision making in a timely manner. Similarly, it requires an understanding of the component relationships and interdependencies … e.g., what if I stop A, what if we speed up B, where to cut funding in bad times, and so on.” The concept of outcomes as leading indicators brings the whole topic of benefits realization closer, as programs can be defined around business outcomes that represent leading indicators. For example, a program could be defined to reduce the cycle time for preparing a proposal, a business outcome that can act as a leading indicator for another business outcome: percent of proposals won, a lagging indicator and definitely a good predictor of revenue increased, a financial outcome. Once again, this reinforces the need to understand causality relationships.

The achievement of financial outcomes, even business outcomes, usually depends on many streams of causality, which make the estimation and the verification of benefits impossible. Gregory Ellis, PM at Dell, Japan, says, “It’s all the talk these days about how PMOs can no longer satisfy themselves with delivering specific business requirements on scope/budget/schedule, but have to take ownership on the business outcomes, blah blah blah. Baloney. This notion fails because of the same reason that attempts to apply Six Sigma to project management fails: there are too many uncertainties and variables at play to offer concrete visibility and control. Did the Sales department’s revenue increase 20% last quarter because you deployed the Sales Force Super-Duper Empowero-matic, or because of the staff augmentation and new Center of Excellence training program that HR implemented, or because of the flex-time, or because of Product Development’s unveiling of a popular new wammy-jammy that captured market share? Ugh … maybe all of them? None? It’s a pipe dream that the Board has thought up because it wants concrete answers on whether its investments are providing raw, bankable value, and a dream that is not rational.”

One more element in this discussion is the topic of baselining. Kiron Bondale, Project Portfolio & Change Management Practitioner in Toronto, Canada, states: “Oftentimes, no baseline exists against which to compare the benefits. In such cases, the project should be responsible for identifying the appropriate metric, gathering a baseline and then delivering benefits against that baseline.” Everything in benefits realization management is incremental, cash in and cash out should be the foundations of a business case, and if there is no baseline, verification of benefits cannot be done.

Conclusion

The compilation of the different points of view regarding why benefits management is so difficult to do seems to converge in two main conclusions:

Strong governance with clear accountabilities and a defined framework are needed, and this has a cost that is not trivial
There is an intrinsic problem in measuring and managing benefits due to causality relationships that makes it difficult, if not impossible, to do

The first conclusion was easy to anticipate. The second points to the need for innovation and to do things in a different way. There is basically one way of doing portfolio management, and it doesn’t work most of the time. There is a parallel to the use of waterfall in the past for every project. It worked for some of them, but it never worked for those projects with high volatility and uncertainty, in which creating a schedule or trying to manage from it was an exercise of futility. In a similar way, there is a need to do things in a different way in benefits realization, because the current approach, which was developed for a more stable environment where most projects were operational in nature, is no longer the norm but the exception. Throwing governance on top of an approach that doesn’t work is not going to make things better; on the contrary, it will highlight the shortcomings and put the whole concept of benefits realization management at risk.

The main points from this discussion have also been used to produce a short video called “Benefits realization, the problem to be solved”: This video also includes an alternative approach: top-down portfolio management, which I have proposed in other articles, in which benefits are estimated at the portfolio level and as a result of the implementation of a strategy or plan. These benefits are then allocated based on the relative contribution of each initiative to the intermediate and final results. In this approach, benefits realization is focused on the achievement of business outcomes, which are more tangible and easier to measure, making benefits realization more effective and easier to implement.

To summarize, the area of benefits realization management is an accepted concept in most organizations, and sometimes even a mandate for heads of a PMO. However, there is a gap between intention and execution. Throwing governance behind this could make things even worse. The reality is that until the intrinsic limitations introduced by causality relationships are addressed, this endeavour will neither be realized nor beneficial.

Don’t forget to leave your comments below.

A Top-down Approach to Portfolio Management

Written by Fernando Santiago on October 5, 2011. Posted in Articles. 2 Comments on A Top-down Approach to Portfolio Management

During the nineties, we were shocked by the Chaos Report with stats that showed two thirds of projects were failing. After fifteen years and the introduction of new approaches including agile, project delivery has improved. However, there are still fundamental problems that have not yet been addressed.

Despite significant investments in technology and organizational change, companies fail to make the connection between strategy formulation and execution. This is evident as well at the government level where failure to implement strategy has the economy surfing in and out of recession. IT projects are a key component of implementing strategy, and the stats are dismal:

Kaplan and Norton tell us that nine out of ten companies fail to implement strategy
According to IBM, 40% of investment in IT is wasted
Gartner puts this estimate at 20%, equivalent to $600B/year
If we take IBM’s estimate, we are talking over one trillion dollars a year, wasted

If companies are failing to implement strategy and wasting fortunes in IT, there is clearly a problem in portfolio management. The mainstream practice uses a bottom-up approach:

Put ideas through a “funnel” process, based on a business case that gets progressively elaborated
Rank opportunities for investment based on their value, risk and alignment with strategy. This sometime involves complex multivariate analytic tools that few people understand or trust
Select the top of the list for execution, all in a yearly cycle
Confirm the benefits in the business case after delivery

While this approach is logical and sound, in many occasions leads to unexpected results:

Business cases based on fiction and not used for decision making
Executives that bring a last minute list of projects “on a napkin” that jump the queue and get approved
The verification of benefits becomes an elusive task and just doesn’t get done
Millions are invested and many times there is no clear idea of what the organization is trying to achieve

It isn’t surprising that portfolio management has not succeeded in helping companies implement strategy. The term portfolio comes from finance and leads to the root of the problem: the assumption that opportunities for investment are independent, and that each one generates measurable financial returns. Finance departments have dictated the need to come up with “hard numbers” and a rate of return for every project. This approach works well when projects represent incremental improvement of an existing system that is not changing significantly. As an example in manufacturing, a new press will cut scrap by 20%, you put the new press in and measure scrap, end of story.

The problem exists when there is a strategy for transforming the business. As Porter states in his article What is Strategy: “While operational effectiveness is about achieving excellence in individual activities, or functions, strategy is about combining activities”. The key word, in Italics in the original, is combining, as it goes against the concept of independent opportunities for investment. In a transformational strategy, activities are represented by projects that deliver capabilities, which then interact to generate results and sustainable competitive advantage. In his article, Porter presents Southwest Airlines as an example of strategy. In a nutshell, in order to offer lower fares, SW offers no meals, baggage checking or seat assignment and operates short flights from small airports. This allows SW to reduce gate turnaround time and increase plane utilization. In addition, it uses only one model of aircraft to reduce maintenance costs. As a result, SW can offer lower fares than their competitors and point to point flights, and after more than 20 years they are still in the market and thriving. Talking about SW, Porter affirms: “Its competitive advantage comes from the way its activities fit and reinforce one another”.

In the case of Southwest, would it make sense to create a business case for “not checking luggage”? How much of the reduction in turnaround time at the gate can be attributed to this? Of course you can come up with a number, but it would be a guess and impossible to verify. Most importantly, in order for the strategy to work, all of the key components need to be in place, so creating business cases and prioritizing a list doesn’t work in this scenario.

The alternative I am proposing in this article is a “top-down” business case approach using a Results Chain model that captures the interactions between financial outcomes, business outcomes, capabilities and initiatives. This approach applies to organizations that are planning a transformation, as opposed to incremental improvement. The top-down business case estimates the benefits the organization will realize from transformation; incremental to the results it would obtain using the current business model. The difference between the transformed and the current scenarios generates a stream of benefits for the overall business case for transformation. These benefits are then propagated through the Results Chain based on the relative contribution of each outcome, capability and initiative. Sounds simple? It is; the elegant simplicity of this solution draws a parallel with the alternative provided by agile to the problem of creating a schedule for projects difficult to estimate, by eliminating the schedule and using relative estimates in points. Today most of us accept this drastic approach, which has proven effective in many situations; a simple solution for a complex problem.

Continuing with Porter, Figure 1 is my personal interpretation of the Southwest case study. The overall benefits of implementing the strategy are estimated in $100M, which are propagated from right to left, based on the relative contribution of each element. In the first propagation, 30% of the $100M goes to Revenue Increased, a financial outcome. These $30M are then propagated to Average fare reduced, a business outcome and from there to Average cost per seat reduced. A second flow starts from the right with $40M propagated to Variable Cost Reduced and from there to Average cost per seat reduced, which adds to the previous $30M for a total of $70M. What this is telling us is that 70% of the benefits are attributable to this business outcome and, as such, should be a focal point for the transformation.

Fernando1

Figure 1

In a similar way, investments for the initiatives are propagated from left to right, and $60M in investments are required to deliver the key business outcome Average cost per seat reduced, which has benefits of $70M, and this should yield a positive return on investment.

If you find this approach too complicated, there are tools that can handle the propagation and calculation of financial returns and allow you to concentrate on what is really valuable: translating the strategy into measurable outcomes, understanding (and not ignoring) their interrelations, identifying the capabilities to achieve those outcomes and the initiatives that will deliver those capabilities. The key for this approach is the identification of the proper business outcomes, which in essence “translate” the strategy into actionable and measurable results. All the fluff is removed from the strategy and the organization can focus on achieving those key business results and not just on delivering projects.

Going back to the business case, in this approach it makes sense only at the overall results level, by comparing the incremental benefits from transforming the operation and the investments to get there. The benefits and investments at each node in the Results Chain serve only as a reference to allocate and manage budgets based on the contribution of each element to the overall results.

This may appear as a simplistic solution. However, if we draw another parallel to agile, we find similarities in the fact that, as agile resolved the problem of predicting the future on a schedule based on hard estimates by eliminating the schedule and using relative sizing, this approach removes the need to come up with business cases for every project, and replaces it with relative contributions. Agile works well in projects that have high volatility and where hard estimates are impossible. Similarly, this approach works well in transformational strategy, where interactions make the estimate of benefits for individual projects impossible. In other words, the options are: create business cases based on fiction for those key projects that will transform the company, or look at the transformation with a holistic view, and manage the portfolio from the top.

By taking portfolio management to the next level of translating and executing strategy, this approach provides PMOs with the opportunity to reposition themselves in the organization and move towards a Strategy Management Office, as described by Kaplan and Norton, which focuses on execution of strategy. For consultants, this represents an opportunity to provide their customers with a fresh approach to implement strategy, and if nine out of ten companies are failing at this, and one trillion dollars are wasted every year just in IT, the business opportunities for everyone are significant, and this proposition does have a clear business case.

Don’t forget to leave your comments below.

Fernando Santiago has over 20 years of experience as a project manager/director and PMO manager in Canada, the USA, Latin America and Europe. He is a solid practitioner, consultant and trainer in the areas of PPM and strategy implementation. Fernando leads P3M Consulting, firm that provides training, consulting and develops PPM software applications.