Lessons Learned: PMO Audit
Author: Charlie Wright
I recently participated in an incredibly interesting audit of a very large information technology (IT) service provider’s project management office (PMO). I’ve performed dozens of project audits throughout my career—and audited a few PMOs—but this was the largest and most impressive PMO. As an auditor, I gained insight and learned a number of lessons from this audit. Here are some of my observations.
Our client has been developing off-the-shelf software for more than 40 years. It now supports more than 9,000 organizations ranging from de novo to multibillion-dollar companies. The client routinely develops in-house and outsourced core processing software internally and markets it to customers, with core offerings that integrate with more than 140 complementary products and services. The company’s impressive development team created an effective project management methodology and strong PMO to manage and oversee projects. It has more than 200 dedicated project managers spread throughout the company and around 20 employees in the centralized PMO.
The client’s chief audit executive approached BKD and asked us to review the controls and processes employed by the PMO. The company uses Control Objectives for Information and Related Technologies (COBIT), the internationally recognized framework developed by ISACA®, and spent a lot of time mapping and cross-referencing various other control frameworks to COBIT. While COBIT is designed to identify the IT processes and controls to help IT align with company objectives, it also can be used for other parts of a company that require structure around controls.
Due to the PMO’s maturity, we immediately knew we’d need to use the Project Management Institute’s (PMI) “A Guide to the Project Management Body of Knowledge” (PMBOK® Guide) – Fifth Edition as our control framework for this audit. The client also asked us to cross-reference the PMBOK Guide controls to COBIT as part of our project deliverables.
The PMBOK Guide recognizes 47 process groups that fall into the following 10 knowledge areas:
- Project integration management
- Scope management
- Time management
- Cost management
- Quality management
- Resource management
- Communications management
- Risk management
- Procurement management
- Stakeholder management
Our audit test program closely followed the knowledge areas. We took a judgmental sample of projects and reviewed each project for evidence the controls identified in each knowledge area were properly designed and effectively operating. For example, PMBOK Guide process 4.1 requires the development of a preliminary scope statement. Our test step was to “obtain and review the preliminary scope statement.”
COBIT is subdivided into four domains and 34 processes in line with responsibility areas of Plan, Build, Run and Monitor. In the Plan domain, process 10.6 specifically focuses on the need to plan and initiate a project using a preliminary scope statement. Consequently, we were able to cross reference PMBOK Guide 4.1 with COBIT 10.6.
Using this technique for each PMBOK Guide process area, and cross-referencing each to COBIT, helps provide an excellent level of comfort regarding coverage. The client uses COBIT as its master control framework and cross-references all other frameworks to COBIT. This level of structure and discipline explains a large part of our client’s success in project development.
Overall, we found this PMO was effectively functioning. It had established a strong project management methodology, and business units were leveraging project management expertise and complying with the methodology guidelines. In general, projects were being completed on time and within budget. And, considering there are dozens of projects being implemented at any time, the project success rate was quite enviable.
However, like any organization, there also were a few opportunities for improvement. We found three for the client to consider:
- We recommended strengthening the controls around cost-benefit analysis. Some projects we sampled suggested that once an initial benefit analysis was completed, there was inconsistency about whether someone was reviewing the actual benefits when the project was finished. Similarly, there was inconsistency in performing a cost analysis to see how effectively project costs had been managed.
- While most projects had filled out a perfunctory report regarding lessons learned, many could benefit from a more robust analysis and intentional focus on lessons learned and information sharing with other project managers.
- It’s important for project management data to be available, consistent and reliable across the company. Since there were a lot of projects in many different departments, there was an opportunity to improve data consistency by leveraging some project management software the company already had.
What I Learned
First, I learned an effective PMO is an invaluable aid in consistently and effectively managing projects throughout the organization. While that doesn’t sound like rocket science, I’ve seen many companies fail at implementing a PMO due to a lack of senior executive support, insufficient resources or attempts to follow a weak methodology.
In addition, integrating and leveraging an established control framework results in a stronger, more effective control environment for the organization. Internal audit at this client was laser-focused on ensuring the business units were following a structured framework. While I’ve always thought of COBIT as an IT framework, I was surprised to find it actually can serve as a good control framework for the entire organization.
Finally, while auditing a specific project is a sensible step in reviewing your project management processes, a more robust, comprehensive review of your overall PMO likely will add more value.
If you have questions about your PMO or how to audit specific projects, email me or call 405.415.3923.