Measuring In-progress Project Performance

Measuring performance is a critical factor in optimizing performance. Optimal performance is sustainably achieving multiple, often conflicting, objectives under changing conditions.

Project performance, on the surface, seems easy to measure; just track time, cost and scope and it’s done. But when we look more deeply we find that it is not that simple.

First, we find that there is confusion about what we actually want to measure. Is it the success of a single project, the success of project performance across many projects, or the success of project management and a project office? If our objective is to measure the success of a single project, are we looking at the project during its life or at the project after it has been completed?

Once we determine what we really want to evaluate we can identify the key performance indicators (KPIs) that will tell us whether we have been or are being successful or not and to what degree. KPIs give managers the most important performance information to enable them to assess the performance of a project or process. Generally we look for from two to five or so indicators. One is not enough for any complex process and project performance is a complex process. Too many KPIs make it difficult to see the big picture clearly. Too few make it difficult to diagnose and correct the issues that get in the way of optimal performance.

Project and Business Objectives

Keep in mind that KPIs are not objectives. They are readings that enable a manager to assess performance towards the achievement of objectives. On a personal level we use KPIs such as weight, blood pressure, and cholesterol level to assess general health. The objective is general health, the measures are indicators that can be used to determine if we are tracking towards the objective, or not.

Project objectives are to deliver goods and services, within time, cost, quality and other constraints, satisfying stakeholder expectations. Project deliverables are used to deliver benefits to satisfy the needs of sponsors and clients.

Note the difference between project and business objectives. Projects deliver goods and services that are generally used after the project has ended to achieve business objectives such as reducing costs and risks, increasing revenues, etc. Projects are initiated to achieve business objectives. Project objectives are a means to that end.

When measuring project success, particularly when the project is on-going, it is necessary to focus on the project objectives and performance against schedule and budget estimates. In most cases, we cannot measure a project’s success in achieving business objectives until well after the project has ended. Further, there are factors that are outside of the control of the project that influence benefits realization. For example, in product development the performance of sales and marketing, market conditions, and other factors impact the degree to which expected benefits are achieved.

Project Performance – Schedule and Budget

Performance measurement during a project is to know how things are going so that we can have early warning of problems that might get in the way of achieving project objectives and so that we can manage expectations. A secondary benefit is information that can be used to improve the planning and performance of future projects.

Typical KPIs are schedule and budget compliance, number of scope changes, number of issues and defects, and stakeholder satisfaction.

There is broad agreement that schedule and budget compliance during the course of the project are essential indicators. Projects must end and completion time is often closely linked to the business objectives that drove the project’s initiation. Tracking to a budget in dollars and/or resource time is a key indicator because it gives us a sense of whether we are performing as we have expected to perform. In most cases, project sponsors and clients are cost conscious. They want to know how much they will spend on the project and they want to know it before the project is over.

Assessing the degree to which the project is tracking to its schedule and budget provides an indication of whether the team is going to meet stakeholder expectations (a critical objective in any project). It highlights the need to look into the causes of variance. Causes may be poor estimating, loss of resources, price changes, underperforming resources, too many changes, errors, omissions and defects, etc. Once the causes are understood a course of action can be decided, either accepting things as they are or making changes to remediate any problem that has been identified.

As in earned value management we need to combine schedule and budget perspectives to get a true sense of overall project health. Do not rely on these measures independent of one another, a project can be under budget because it is behind schedule, because prices have gone down, because performers have used clever means to get their work done less expensively, or for other reasons. A stakeholder who is budget oriented can easily get the wrong idea of project health by looking at stand alone budget data.

Quality Deliverables

To effectively measure performance it is necessary to plan so that tasks are defined in terms of specific deliverables with clearly stated requirements. The availability of an accepted deliverable is the only measure of the successful completion of a task. It is all too easy to deliver something that doesn’t work or meet requirements.

Agile project approaches use Velocity as a KPI to measure the team’s rate of progress. Velocity is the number of features (or use cases, components, deliverables, etc.) delivered versus the number planned. This is clearly measuring against schedule but highlights the need for linking tasks to concrete, useful deliverables. Whether or not you are using an Agile approach, it is a best practice to ensure that task completion is defined as the delivery of a specific and meaningful deliverable.

Effort and Cost Tracking

Whether you use velocity or earned value you need to account for effort and the cost of other resources to monitor budget compliance and to estimate to project completion based on current performance.

Can you measure project performance without tracking effort and cost? Of course you can. It is done all the time. However, without a sense of the effort/cost being expended, any assessment of schedule compliance is overly subjective, it is a guess. Capturing and using effort and cost data is often difficult, requiring appropriate tools and cultural change, but if you want to manage your project effectively, do it.

Project Performance – Other Indicators

Other indicators, aside from schedule and budget performance, monitor the number and types of issues, changes, and/or defects and the degree to which they are quickly addressed.

Issues are questions, disputes or problems that arise during a project and that must be addressed to satisfy stakeholders and ensure that the project is heading in the right direction. Issues vary in priority, complexity and the amount of time and effort they require. Issues are inevitable, plan for them and track the effort required to address them against your estimate.

Scope changes result from requests for change in requirements. Changes require analysis, decision making and execution. Scope changes, particularly those that occur late in project life, are disruptive. When planning the project it is best to estimate time, effort and cost to create a fund for expected changes and then to monitor actuals against this fund.

Defects are discovered when testing is performed to validate deliverables. Defects require effort to determine their cause and correct or accept them. Again, it is best to estimate defects and their impact in duration and cost and to track against this estimate.

Each of these is a KPI. High instances of each indicate that there will be project schedule and budget slippage. Issues, changes and defects should be tracked and aged to give management a sense of what is happening outside of the schedule and budget.

One of the most important and underutilized project performance indicators is stakeholder satisfaction. During the course of a project assess the degree to which clients, sponsors and performers are satisfied with responsiveness to their issues, the sense that progress is being made, the degree to which they are involved, the health of relationships and their general feeling regarding the project’s performance.

Conclusion

This article has focused in on performance measurement of in-progress projects. The objectives of performance measurement are 1) to enable project teams to manage stakeholder expectations by informing stakeholders of where the project is with respect to the plan and where it looks like it is headed, 2) obtain early warning of problems that might affect the overall performance of the project and diagnose the situation to determine the most effective course of action.

Note that achieving business benefits is not a practical KPI for projects while they are in progress. Benefits do not accrue until after the project has delivered its deliverables and they have been used.
The key performance indicators discussed are schedule and budget compliance, issues, changes, defects, and stakeholder satisfaction. Combining these into a dashboard and drilling down to assess their causes and impact helps to ensure project success.

Don’t forget to leave your comments below.