The Death of the IT PMO?

Earlier this month, I attended a session hosted by our PPPM Community of Practice. During the session, I was surprised to see the old gem: “The Standish Group Report” on the state of IT Project Management. (More commonly referred to as “The Chaos Report”). This report has been published since the mid 90’s and is routinely [ab]used to support every premise from “Fire all of your PMs” to “Turn over the reigns of the company to your PMO”.

Admittedly, the Chaos report is a very interesting piece of research and the details are extremely useful for identifying areas of concern. But the summary statistics that are generally quoted have a bad habit of leading people to make some seriously flawed assumptions about IT project management

The Chaos Report Summary

The Chaos report categorizes IT projects into three “buckets”;

• Resolution Type 1, or project success: The project is completed on-time and on-budget, with all features and functions as initially specified.
• Resolution Type 2, or project challenged: The project is completed and operational but over-budget, over the time estimate, and offers fewer features and functions than originally specified.
• Resolution Type 3, or project impaired: The project is cancelled at some point during the development cycle. (referred to repeatedly throughout the report as “failure”)

These numbers are routinely used to show the lack of maturity in IT Project management as compared to other industries. Notice that none of these criteria address business value, customer satisfaction or the return on investment. The criteria also rely on “initial” specifications and “originally specified” features and functions. Finally, these figures treat project cancellation as  “failure” with no regard for the circumstances.

Should we be concerned about the state of IT project management? Are more projects truly “failing” than in 2004?

Information Technology Projects are different

The main problem with these numbers is the underlying premise. IT projects are *not* construction, civil engineering, public works or other long-term projects.

While a bridge, shopping mall or sewer system may remain viable for decades or even centuries, a software development project is likely to become woefully out-dated in 18-24 months.

Changing market conditions and technologies are unlikely to fundamentally change a construction project “in-flight”. Unless an earthquake moves a landmass, the design for a bridge is likely to be just as viable in 10 years as it is today. However, a new competitor in the market place, a new technology, a major version upgrade in software or even a change in business processes could completely derail an IT project literally overnight. Something as simple as a new smartphone technology could completely reset customer expectations and drive major requirements or strategy changes in completely unrelated industries. The Internet/iPhone/Twitter/<insert technology here> didn’t change the way that a road gets built, but they certainly disrupted business models and internal IT infrastructures. While a new car design, drug, toothpaste, etc. can fall slightly out of step with current technology, an IT project may end up being the equivalent of a half-completed steam engine while the competition is soaring around in it’s new hovercar.

I’m certainly not saying that IT projects can’t be managed, only that they need to be measured differently and we need to be a little more selective in how we define “success”.

Deconstructing The Chaos Numbers

“Success” as defined in the Standish Report means that all three reported criteria (budget, schedule and adherence to initial requirements [Standish’s definition, not mine]) are right on target. Mathematically, a 32% “successful” rate means that the average accuracy of the initial estimates is somewhere around 70% (70%^3=~34%). I don’t know about you, but in the IT projects that I’ve been involved in, this is a remarkable hit rate for pre-design requirements. What scares the hell out of me is that this would mean that these same 32% of IT projects don’t change *any* requirements based on customer feedback, changing technology or evolving market conditions throughout the project. Either the respondents lied, the communications processes in the projects are broken or projects are launched and essentially run in a vacuum. None of these are “good” project management.

The 24% cancellation rate doesn’t concern me since it’s based on “number of projects” and not on project costs. Unfortunately, the report doesn’t address the sunk costs, the conditions for cancellation or how early in the process these cancellations occur. If you’ve got a decent mix of R&D or speculative projects, having 25% of them killed early just indicates that you’ve got a healthy portfolio management process. IT should be working with the business and experimenting with new technologies to assess the business value. If the value isn’t there, the projects should die. In addition, projects *should* get cancelled as they’re merged, restructured or made obsolete by rapidly changing technologies, business needs and priorities. The biggest problem with most IT shops is failing to cancel projects that live beyond their value window. Instead, they blindly follow the “on time”/”on budget” mantra and plow through to a “success” that doesn’t include the delivery of any actual business value. It’s also not at all uncommon for IT to work on “contingency” or “risk mitigation” projects in parallel with more speculative, but preferable solutions. Once the more risky project is proven out, the contingency projects stop. Construction crews rarely have a need to build a “backup house”. They certainly rarely have a need to experiment with an unknown material or technology (since the building codes don’t tend to encourage that sort of experimentation in the field). Unfortunately, software and hardware vendors aren’t held to the same standards and interoperability rules that suppliers of construction materials are. (“Sorry, we have a bug in our steel girders that cause them to buckle if used anywhere near that brand of drywall”)

That leaves us with the 44% in the category of “challenged”. Again, this is completely to be expected. There’s a good reason why “Agile Development” has never become popular in construction. In construction, the requirements are generally well defined. The tools and materials are well proven and understood. The design and engineering is often done as part of the bid process and the average “customer” has a respect and trust for the skills and expertise of the people hired to do the job. It’s rare that someone having a house built for them dictates the type of bulldozer, number of nails or the source of concrete. It’s also rare that the customer calls for the foundation to be torn out and rebuilt after the second story is erected or expects the contractor to ignore the local building codes and best practices. I also seriously doubt that any contractors would stand for the customer demanding that major portions of the work be done “in the margins” without any additional funding. Yet analogous situations arise in IT every day. Developers are expected to use unfamiliar tools because of golf-course agreements with vendors. Project Managers manage minutiae and task details that have little or nothing to do with the actual project delivery and fundamental requirements can change overnight. A considerable amount of this is the fault of IT for not integrating more closely with the business and earning the respect and trust of their partners. But a bigger part of this is simply the nature of IT projects. They have a very narrow window of opportunity, the deliverables have a very short life expectancy and both rely on a technical and market landscape that moves incredibly fast. Unless you separate “business requirements” from the functional and technical requirements, it’s naive to assume that they’ll be anything close to “fixed” at the start of the project.

Today’s PMO

In the 2008 PMI Study Researching the Value of Project Management, there was a very telling observation that many organizations had, or were about to, reach an “inflection point” in their PM implementations. They had achieved a certain level of value, but were failing to achieve any additional benefits. In some cases, they were actually seeing a decline in value. I’ve seen this repeatedly in my own experiences as PMs (and more specifically PMOs) become entrenched in policing process and lose sight of their role as project facilitators. The tools, process and policies become more important than delivering business value. So, instead of being the enablers and advocates for project teams, they rapidly become the 1984-style “Big Brother”, watching every move, monitoring schedules and budgets for discrepancies and punishing the offenders. The Triple constraint becomes budget, schedule and filing your status report on time. No consideration for quality or customer satisfaction remains and “success” is measured by the amount of paperwork you can generate and coming in on-time and on-budget. The finished product can be completely non-functional, but you hit your targets.

• Issue 1:  In a typical IT PMO, the majority of its time is spent tracking, monitoring and reporting on projects to the stakeholders, clients, sponsors and executive leadership. However, this particular aspect of Project Management was reported as having the *least* impact on Project performance (particularly in manufacturing and IT/Software Development organizations). The respondents included Project Managers, team members and the actual intended audiences of this reporting. If the primary activity has the least impact, why are we doing it?
• Issue 2:  The PMO is often the owner and manager of the Project, Program and Portfolio management toolsets. If you take a look at any of these tools, you’ll be hard pressed to find any of them that include “Quality”, “Customer Satisfaction”, “Business Value” or “Meets Requirements” as metrics attached to any task or report. Instead, you’ll find a dozen different ways to report on “Budget” and “Schedule”. Is it any wonder that these become the focus as the PMO matures and starts relying on these tools instead of actually talking with the PMs and customers? It’s also no surprise that many IT PM teams give themselves high-fives and celebrate successful projects while the customer is left with semi-functional tools and the impression that IT is a bunch of overpriced idiots.
• Issue 3:  The one area where most PMOs seem to have gotten it right is in driving consistency in Project Management practices. Unfortunately, those practices very often ignore the reality of the environment. They follow the traditional “stable requirements” model and the project managers on the floor end up keeping two sets of books; One to keep the PMO happy and one to actually manage the project. Highly talented PMs with strong communication and negotiation skills end up shielding the project teams from “the big bad PMO”, keeping the project running, keeping the customer happy and in the loop and meeting what appear to be arbitrary reporting requirements of the PMO.

So why do we continue to promote a structure that appears to provide little or no value to the stakeholders? In extreme situations, the PMO is actually perceived as getting in the way of successful project execution. Is the IT PMO Dead?

Rethinking the PMO

I’m not going to differentiate between the Portfolio Management Office and the Project or Program Management office. Where they exist as separate entities, they all share these underlying goals. Only the specific scope and focus changes. Where only a Project Management office exists, they tend to have unofficial responsibility for the other areas as well.

At the heart of the PMO, we have one main goal: Maximize the business value of the project portfolio.

In addition we have Five responsibilities that support this goal:

1. Minimize or mitigate risk to the customer and the organization as a whole
2. Ensure customer satisfaction
3. Reduce complexity
4. Drive consistency
5. Listen and Observe

If we go back to this foundation and assess our current practices, it can change a lot of our bad behaviors. Notice that there’s no mention of “control”, “report”, “enforce”, “hold meetings” or any of the other negative stereotypes that the PMO has become infamous for.

Responsibility 1: Minimize or Mitigate Risk

Notice that this responsibility isn’t “avoid risk”. Almost all opportunities present some element of risk. Usually, the greater the potential reward, the greater the risk involved. It’s the PMOs responsibility to do everything possible to ensure that the decisions to accept risk are well-informed and that risk is managed to prevent it from getting out of control. Ideally your culture should reward risk taking and avoid punishing failure. The PMO can have a big influence on driving that culture by demostrating the benefits of openly communicating and managing risk. Repeat after me: “Risk is not a bad thing. *Unmanaged* risk is.”

This ties into the whole issue of “project failure” being a bad thing. The project portfolio should be structured in such a way that the rewards from the successes cover off any sunk costs of these “early termination” projects. It should also be the responsibility of the PMO to help determine when we’ve squeezed all of the potential benefit out of a project and then shut it down before we get too far down the loss curve. Anything that can be salvaged or reused needs to be shared. The PMO also needs to help maximize that benefit by recording and sharing the lessons learned from an early project termination. Early termination is preferable to “successfully” completing a project that ends up providing no business value (or negative business value). Too often, early termination is viewed as “failure” and the project stakeholders plow forward to avoid the negative stigma of project failure. Get rid of the stigma around failed projects and drive open and accurate reporting of reality. Reward teams that cut their losses and salvage value out of a project that would otherwise become a burden to the company. Pharmaceuticals and research labs “get” this. Only a relatively small percentage of projects will ever result in a marketable product. They don’t sit frozen in the headlights until a clear winner emerges in the marketplace or with a competitor. Instead, they treat all projects as learning exercises and cut them off when they stop providing any new value. The value comes from the lessons learned, keeping options open and mitigating (not avoiding) risk through a diverse and forward-thinking portfolio. One of the surest ways to kill innovation is to ignore overall business value and focus strictly on financial ROI and risk avoidance.

Discourage the “Mega Project”. Wherever possible, break down projects into the smallest discrete groupings of business value that you can. If you can fund 30 days of work and create a small but tangible business value every 30 days, it’s considerably less risk than if you have to wait 2 years to get “the big reward”. Small, iterative development cycles make it easier to control risk, dynamically adjust requirements and to continually demonstrate business value. If you completely screw up a 30-45 day cycle, it’s significantly less expensive than if an early bad assumption derails 2 years worth of development. In addition, it’s very easy to allocate a fixed pools of resources and funding for a short period of time (typically 4-6 weeks) and allowing the team to negotiate the deliverables with the customer as they go. Success is easily measured and customers generally have a higher tolerance for pushing requirements to the next cycle when it’s only a few weeks away. You don’t need to fully embrace Agile methodologies or practices. It’s great if you can, but this can work using a traditional waterfall approach and tools with smaller iterative cycles. (spin it how you need to. Your engineers and developers are probably doing this anyways. Align your funding and project management models with reality and everyone will be a lot happier)

Responsibility 2: Ensure Customer Satisfaction

Even if you’re part of a captive IT group, customer satisfaction should be your primary success criteria. A focus on continual customer feedback and communication helps to keep expectations in sync, helps to keep the team focused on what’s important to the customer and reduces unnecessary rework later in the project. In many cases, project managers become so obsessed with budget and schedule that they’ll miss opportunities to dramatically increase the business value of the project to the customer. Instead, they deliver what was agreed to, but not what the customer wanted. On the opposite extreme, scope creep and cost/schedule overruns can result, but if the lines of communications are open, it’s much easier to negotiate changes that work for all parties. If the customer is OK with additional cost, changes in schedule or reprioritizing functionality, it’s the PM and PMOs responsibility to ensure that the customer’s voice is heard. Instead, many PMOs will blindly adhere to the baselines and refuse to accept any changes. By all means, use good project change management practices. But, don’t avoid change on principle.

Budget and Schedule are valuable indicators of the level of project control and health, but they have very little to do with project success. If the business value exceeds the cost and the schedule doesn’t pose an issue for the customer or create resource problems on other projects, it’s somewhat irrelevant if the project is on time and on budget. These metrics are only important if the customer says that they’re important. Unfortunately, as mentioned earlier, none of the project or portfolio management tools seems to take that into account. It’s easy to measure budget and schedule. It’s much more challenging to manage quality and customer satisfaction.

Responsibility 3: Reduce Complexity

This is an area where many PMOs fail miserably. In an effort to enforce consistency, provide visibility and gain additional control of projects, the PMO layers on processes, forms, tools and administrative deliverables. Very few organizations implement exception processes to allow these artifacts to be bypassed when they don’t make sense. Instead, Project managers find themselves completing paperwork for the sake of a checkbox or creating artifacts that are nothing more than creative fiction to keep the PMO from raising a red flag.

Ask yourself what business value your report/tool/artifact/meeting/process has. If the answer is “none”, get rid of it. If you’re managing details that don’t matter, stop doing it. If your PMs are producing metrics and reports that aren’t actionable, tell them to stop. Along a similar vein, encourage PMs to take a similar attitude and stop micromanaging resources. Track the critical path. Track deliverables and dependencies. But stop padding your project plans with tasks that your team doesn’t need you to track. Spend that time talking with your customer or clearing roadblocks instead of tracking minutiae. Trust your team and work on adding value instead of reducing theirs. I was once part of a 60-day/20 person project that had a 3,700 line project plan associated with it at the PMO level. At least 1/3 of those tasks were related to administrative overhead. The engineers had a shadow plan on a spreadsheet which listed the 30-40 deliverables, owners and target dates. (Only 10 of which were on the critical path). It took two full time PMs to maintain the PMO-level plan and attend meetings while the actual work was managed by the engineering lead using the spreadsheet.

Responsibility 4: Drive consistency

Consistency doesn’t mean that you create a robot army of PMs that mechanically manage the tiniest detail of every project. Consistency should be driven by providing a basic framework to work within. I often describe it using the “good fences make good neighbours” analogy. Define the roles and responsibilities. Define your handoffs and then let your teams do what they need to do in their own back yards. Standardized tools, forms, processes, policies, etc. can be helpful guides. But it’s much more important to help your PMs address the intent of an activity than it is to have them fill out form 617J using a number 2 pencil. As you review your existing processes, ask yourself if it’s the PMO’s responsibility and what business value would be lost if you didn’t complete a particular field, form or task. These tasks can still be included in your checklists, processes or methodologies, but make sure that they’re clearly identified as mandatory, recommended or completely optional.

One of the keys to effectively driving consistency is, ironically, a formalized exception process. The inability to “officially” bypass things that don’t make sense leads to the team losing respect for the entire process. If one part clearly doesn’t apply, why bother with the other parts? Allowing and tracking exceptions is a vital tool in streamlining your processes and methodologies. It also empowers your resources to use their own judgement while making them accountable for the decisions. If you know where the exceptions are occuring, you can track them, assess the risk and decide if something needs to be done to mitigate the risk. If the exceptions are buried or hidden, the process rapidly falls into chaos. Additionally, you’ll find that making it easy to get an exception actually drives more accountability. “Sure you can have an exception. Just record the reason, sign here and you’re done. We don’t even have to approve it.” People know when they’re being lazy and they also know when something really doesn’t make sense. The lazy ones lose the excuse that “the process is broken” while the responsible ones have the flexibility that they need to focus on delivering business value.

Responsibility 5: Listen and Observe

Stop talking and start listening. Engage your PMs, customers and project team members and find out what they need to be successful. The PMO doesn’t exist for it’s own benefit or even that of the executives. It exists to help make the most effective use of limited resources for the good of the company. By helping your project teams be productive, you increase their value and the value of the projects in your portfolio. By clearing obstacles and reducing administrative overhead, you reduce the costs of execution. By engaging your teams in discussions of Quality and Customer satisfaction, you ensure that what you deliver is going to be the best possible outcome for your customers. The PMO should be an advocate for the Project Team and customer, not a looming threat hanging over them and not some bureaucracy that does nothing but generate reports all day.

So, is the IT PMO dead?

Some of them may be on life support, but they’re still hanging on with the potential to add a lot of value. However, many organizations may need to rethink how they’re positioning and using their PMO. If you want to get the maximum value out of your projects, then the PMO needs to be focused on value creation. If you focus them on generating reports, processes and paperwork, that’s what you’re going to get as an output. If your PMO is focused on risk avoidance, they’re going to stand in the way of a lot of opportunities. Think about what you want to get out of your PMO and then empower them to make it happen.