Validating Project Management ROI: Looking Beyond Skill Sets and Competencies

This article has been written to illustrate methods that can be used to validate a Return on Investment (ROI), looking beyond skill sets and competencies.

Generally, organizations that engage in a formal Project Management Training Program will look to validate their investment by initially assessing a marked improvement in individual participant skill sets and competency advancement. This is, however, only an initial benchmark of effective project management training and the integration of raised personal attributes and cognizance.

The real test of the validity of a program lies in the participants’ ability to return to the project management environment and make a day-to-day difference in their actual project assignments. This can be measured, stratified and sold by utilizing two different mindsets/methods.

The first mindset/method, best used by an organization that is employing a definitive and mature methodology (defined as a methodology in operation and successfully in use for more than 2 years), would be to target the areas that comprise their key process indicators (KPIs). These areas would be determined by tracking actual project data over the methodology maturity life cycle, i.e., bolstering the skill and competencies that would
make the most positive difference in day-to-day implementation of the definitive methodology.

The second mindset/method would be to utilize the training program content (driven by pre- training interviews) to highlight areas in which the organization is currently taking the most hits on projects. This would include the full range of time, cost, human resource and management issues. The training, in this case, Advanced Training, should target specific solutions to these hot spots and, in effect, create the initial benchmark or key process indicator (KPI) targets against which to track. In doing so, the training would go beyond advanced general topic training to customized course(s) for implementation of the required process steps espoused by the organization.

This would serve as the beginning of a continuous improvement mechanism (the methodology) that can be owned by the project management resources.

Typically, organizations categorized as Level 1 or 2 in a PM-Capability Maturity Model (CMM) schema (either formal or informal), usually have not been tracking projects to a sufficient depth to create any form of accurate and meaningful benchmark/baseline data. The training program development schema, as designed above, would help to establish the initial target baseline that should be monitored at intervals determined prior to the training.

A few scenarios may arise from this approach:

Minimal improvement is detected at the first few milestone/review intervals across a wide range of pilot projects, and the participants are at a loss to explain the situation (no increase in cognizance). This would require an examination of the program goals developed from the participant and management anecdotal experience. The current day-to-day situation on the projects would also be reviewed (i.e., are the scenarios/hot buttons accurate and in alignment with those utilized as course drivers? Did we utilize the wrong targets or the wrong opinions/input?)

Improvement has been detected at the first few milestone/review intervals across a wide range of pilot projects. The participants have additional information as to where there are “roadblocks” (process or organizational) to further applying the skills the program targeted and transferred (increase in cognizance). This situation, although not exhibiting marked or substantial improvement, is very favorable, as the participants have
been active in the implementation of the program’s information. Through a level of raised cognizance, they can identify improvement opportunities ahead of them.

• That being said, one must then analyze the improvement opportunities for real and substantive issues versus the possibility of identifying further excuses for lackluster performance. If you are working with an outside entity to assist your efforts, it is important you are involved in this process, as you know your team best.
• An important piece to mention is that once the real and substantive issues have been identified, the organization should develop a positive response and implementation strategy, including a timeline, to clear them up. If not, the effort will be unsuccessful as the participants may feel as if “they [management] were trying to fix us rather than educate us to recognize what is and is not working.”

The last scenario is that marked and substantial improvement is detected at the first few milestone/review intervals across a wide range of pilot projects and the participants have additional information as to where other improvement opportunities are available (increase in cognizance). As previously stated, we would then analyze these areas of possible improvement for cost of implementation and value of return. This is slightly different from above due to the fact that in this scenario the “bleeding” has stopped, and the project is in far better shape. We therefore could be more deliberate in our approach to the improvement opportunity planning and implementation.

Don’t forget to leave your comments below.