Practical Approach to Project Management Metrics
Abstract
Project management metrics are vital to implementing practical and sustainable project management practices and processes in any organization. The key is to keep the metrics simple, practical and relevant to the organization.
The paper describes metrics implemented at The Bon Ton Stores, and the processes put in place to establish and measure these metrics. This information is useful to anyone who is implementing or evaluating project management metrics in their organization.
The paper touches on the key requirements and challenges in implementing the project management metrics. Then it goes into each metric in detail to discuss what is measured, the data collection process, communication (frequency and audience), challenges faced and controls in place. Actual examples from Bon Ton are included.
Project management metrics are key to improving how projects are managed and delivered, as well as demonstrating year-over-year improvements in project management maturity.
Project Management Metrics Key Requirements and Challenges
Project management metrics help us:
- Measure and understand the maturity of the IS organization
- Manage projects and resources more effectively
- Demonstrate year-over-year improvement in performance
- All metrics should be:
- Simple and make sense for the organization
- Measurable with clarity, fairness to all, and without ambiguity
- Supported by and traceable to real data
Key Requirements
There are certain key requirements and guidelines for the implementation and consistent measurement of project management metrics:
- Repeatable and sustainable project management processes
- Monitoring compliance to processes
- Collection of data using a project management tool
- A performance baseline may take some time to ensure compliance to processes and accuracy of data
- Timely and clear communication of metrics, baseline, and any subsequent changes to metrics or the data collection process
- Don't have too many metrics, especially if they are similar
- Don't have too many exceptions on how the data is collected and how the metrics are calculated
Challenges
The implementation of project management metrics will face some challenges that need to be handled with proper processes and controls:
- As the compliance improves, initially the figures may go down (i.e., metrics may not show improvement)
- If the metrics definition or measurement process is changed, previous years figures would have to be recalculated
- If the base data changes, the metrics change and have to be recalculated
- People may not pay much attention to metrics unless it impacts them individually (e.g., when personnel review was tied to the metrics, the compliance improved significantly)
- If the trend does not look good, the metrics process is suspected (have data to explain metrics and trends clearly)
- Project managers (PMs) resist setting baselines, or, set baseline as late as possible in the project so they are not missed
- When multiple deliverables are involved, people don't want to set baseline until all requirements are defined
- Certain projects pose unique challenges such as: requirements/analysis projects, as-time-permits projects, and internal maintenance projects
- Unique challenges posed by the Retail industry: 4-5-4 Retail calendar; each month is different; A 53rd week every 6 years
Metrics used at Bon Ton
At Bon Ton stores we have implemented the following metrics so far:
- Project Time Metrics
- Project Performance Metrics
- Within 10 calendar days of committed date
- Within 10% of approved budget
- Reason Code Analysis
- Project metrics by Business Group
- Project Scorecard
- Project Satisfaction Survey
Project Time Metrics
This metric measures time spent on forward-looking projects vs. maintaining status quo. It also measures capital hours vs. expense hours.
The key requirement for this metric is that projects are properly categorized as new development (New Product/System or Enhancements), maintenance, support, misc. work activities and misc. non-work activities, as well as designating projects as capital or expense. The actual hours are captured using timesheets in the project management system.
These metrics are communicated to managers and above on a monthly basis.
The challenges include getting everyone to submit timesheets on a timely basis, proper categorization of projects, and the definition of what is forward looking (e.g., while maintenance projects usually maintain status quo, some of them may be classified as forward looking projects).
Controls put in place include appropriate management supervision of timesheet submission and categorization of projects.
Exhibits 1 and 2 are examples of project time metrics report and time analysis by resource.
Exhibit 1 Project Time Metrics
Project Performance Metrics
This metric measures project management performance by PMs compared to their approved budget and committed delivery date. The goal for PMs is to deliver projects within 10% of their budgeted work effort and within 10 calendar days of the agreed upon delivery date. The metrics are rolled up to the various management levels to measure project management performance at each level.
The key requirements for this metric are that projects are baselined at the appropriate time, and finding the true reason for any baseline change. The performance baseline work effort and go-live date are captured by the PMO, and the actual hours are captured using timesheets in the project management system.
This metric is communicated to managers and above once a quarter.
The challenges include getting managers to baseline their projects at the right time (after requirements are defined, design work is completed and a project plan has been developed), assigning appropriate reasons for any baseline change, and the sensitivity of the metric since it is tied to the annual performance appraisal.
Controls put in place include monitoring of project baselines and baseline changes by the PMO and the management.
Exhibit 3 is an example of a project performance metrics report at Bon Ton.
Reason Code Analysis
This metric measures project delays in hours and/or days and number of incidents that caused a delay for each factor (reason) causing project delay. The delays are categorized as user caused, vendor caused or Information Services caused delays.
The key requirements for this are finding the true reason for any baseline change and assigning the appropriate reason code. Each baseline change is reviewed by the PMO and the management, and the appropriate reason code assigned. Actual hours are captured by timesheets in the project management system.
This metric is communicated to senior management every six months.
The challenges include finding the true reason for each baseline change. Some situations are deemed to be within the control of IS, but may not be (e.g., resource availability).
Controls put in place include monitoring of baseline changes by the PMO and the management. Every project change in baseline estimate and baseline date are reviewed by the IS management team to determine if it was within the reasonable control of IS. Appropriate reason code is assigned to the change. A change in date is published to the business system owner and executive sponsor along with the reason why the change occurred.
Exhibit 4 is an example of the reason code analysis at Bon Ton.
Project Metrics by Business Group
These metrics measure the work done by IS for the various business groups analysis of time spent on forward looking work vs. maintaining status quo, committed and uncommitted project backlog, and the # of projects initiated and completed.
The key requirements for these metrics are that projects are properly categorized as New Product/System or Enhancements, Maintenance and Support, and the correct designation of the business group on each project. The project data and the timesheet data come from the project management system.
These metrics are communicated to managers and above once a quarter.
The key challenge is the designation of the appropriate business group for the project. Master project and its subprojects may be assigned to different business groups. The backlog information is only as accurate as the work effort estimate for the projects. Defining what is Initiated and what is Completed has to be clarified sometimes, projects get initiated, but are not completed (cancelled, suspended), and some projects may stay active (open) for a few weeks (sometimes months) after the actual go live.
The control in place is that the VP of applications reviews categorization of projects and business group assigned.
Exhibit 5 is an example of project metrics report by Business Group at Bon Ton.
Project Scorecard
A project scorecard helps us understand the success of a project by measuring the project performance against criteria that need to be met for the project to be a success. The criteria are the expectations defined at the beginning of the project and in Requirements, and when a detailed project plan is developed. It includes Cost, Schedule and Quality Performance, and User Satisfaction.
User Satisfaction is measured by conducting a stakeholder survey. This survey answers questions such as:
- How well did the project team perform?
- Did the project deliver an effective solution to address customer requirements?
- Did the deliverables meet project objectives?
- How satisfied are the users with the outcome of the project?
The project management office prepares the scorecard for each project upon its completion. Exhibit 6 is an example of the project scorecard.
Final Thoughts and Next Steps
The project management metrics at Bon Ton were implemented over the last five years and have given us a way to measure our success in delivering quality projects on time, under budget as per expectations. It has also helped us in demonstrating sustained performance improvement over the years.
Some of our future steps include:
Presentation of Metrics via Dashboards
Expanding metrics to include more financial data
Publishing more metrics to the business users
Measuring the project ROI as part of the project scorecard.
Original Copy:
Sham, K. (2013). Practical approach to project management metrics. Paper presented at PMI Global Congress 2013 North America, New Orleans, LA. Newtown Square, PA: Project Management Institute.
Here's more information about:
"Practical Approach to Project Management Metrics"
Five Project Management Metrics key to Successful Project Execution Operational Excellence
http://obolinx.com/resources/2013/10/five-project-management-performance-metrics-key-to-successful-project-execution-operational-excellence/
Performance Metrics For Areas Of Project Management
https://resources.workfront.com/project-management-blog/performance-metrics-for-areas-of-project-management
Metrics for project success
https://project-management.com/metrics-for-project-success/
-
Activemode Network Mobile Training
Desktop and Mobile Device
https://www.activemode.com
Work / Life Balance Coordinator
https://www.activemode.com/Worklifebalancecoordinator
Skillset Assessment
https://www.activemode.com/Skillsetbalance
30 day Evaluation
https://www.activemode.com/Mobileworkrules