Project Success Criteria: What to Measure

I've often written that we shouldn't focus on measuring people, nor let time and budget metrics be our key indicators of project success. This is a key component of my SOPM (Service-Oriented Project Management) model.

That's not to say that time or budget should be ignored. They should be managed. And where it's a vital element of the project, it should be monitored. But these items shouldn't be used across the board to determine whether our organization is successful with project management.

As for measuring individuals, some claim that it's the only way to get results out of people (i.e. what gets measured gets done). I say this is leading by fear, and the dangers outweigh the results it brings. We can also get results by prodding people with a hot poker. That doesn't mean it's the most effective way to create an engaging environment. Napoleon once said there are two levers for motivating people--interest and fear. We're always better off using interest, whether it's with a compelling story, a contest, a call to action, or some other more positive attraction.

So, what SHOULD we measure? Here are the 5-7 things that are most vital:

1) Client Satisfaction (on an ongoing basis, not just at the end of our project when it's too late for course corrections)

2) Employee Satisfaction (this gives us insite into the culture we're promoting. A good list is to use the 12 questions Gallup has determined that employees ask themselves to determine their satisfaction level, as noted by Marcus Buckingham and Curt Coffman in First Break All the Rules. Then, management can be measured by how their employees answer those questions.)

3) Outcomes (progress on current milestone deliverables and interim results as they are delivered ---even if some outcomes can't be measured until the project is technically over. This focuses on the work completed and its tie to real value -- not just time being ticked off on a clock. Also it's best to measure a team's success against those outcomes as opposed to individuals---or their respective departments---and their discrete objectives. This promotes a team atmosphere in which all collaborate toward common goals.)

4) Cultural Alignment (see www.infotool-online for a good product for diagnosing cultural alignment by demographic, showing how well everyone is aligned on "the main thing" your organization is trying to establish culturally---the product also suggests remedies and a path forward).

5) Vision Alignment (the best tool I've seen for this is in The Disney Way and The Disney Way Fieldbook, by Bill Capodagli and Lynn Jackson. Their VisionAlign model, based on practices taught at the Disney University, is a matrix, listing Vision/Mission key points on the left side, and Core Strengths, Values, Objectives, and Stakeholder groups across the top. The columns are mapped to the Vision/Mission points via correlation indicators indicating a fit, potential fit, or area of concern.)

.... And for the more ROI conscious...

6) Total Cost of Ownership (not just the cost of the project. This would include ongoing maintenance, support, etc., at least 3-5 years out, or for the expected life of the product)

7) Total Value of Ownership (including tangible and intangible benefits, and potential long-term and indirect benefits. Accountants and financial types hate this, and there's no magic formula for it, but to ignore it completely is shortsighted.)

One thing is for sure -- continuing to measure project success in terms of on-time, on-budget, and meets specs is taking a very limited view of what success really is. It's time we redefine it.

I'd love to hear others' thoughts on this, and other items worth measuring that keep our focus on real client value and employee engagement.

Service-Oriented Project Management (SOPM) Podcast Interview

As mentioned, I recently did a webinar on Service-Oriented Project Management (SOPM) for Mindjet, makers of MindManager software. In addition to that, Cornelius Fichtner just interviewed me about SOPM for his latest episode on The Project Management Podcast. It was a fun interview, as Cornelius always has interesting questions. Hope you enjoy it.

Here's the link to the podcast.

The Project Management Podcast™ - Episode 058: Service Oriented Project Management

PS: I'll be resuming my podcast series again shortly as well. I took a hiatus while I was tied up with speaking engagements and seminars over the last few months. Stay tuned for details.

Service-Oriented Project Management (SOPM): Free Webinar and White Paper

For those interested in learning more about the Service-Oriented Project Management (SOPM) framework I introduced here on PMThink several months ago, you may want to check out the free webinar I did with Mindjet software on the use of their MindManager product with SOPM.

As many of you know, I'm a big proponent of mind-mapping software for brainstorming, organizing, and presenting, and it's invaluable when used with SOPM. The recorded version of the webinar is available on Mindjet's webinar archives page (below), and a free white paper on SOPM is available there as well.

Enjoy...

Mindjet: Webinars

Webinar on MindMapping for Client-Focused Project Management

For those that are interested, I'll be doing a free webinar for Mindjet on November 29th (2:30pm EST, 11:30am PST) on the use of their MindManager product, especially as it applies to the Service-Oriented Project Management (SOPM) framework.

If you recall, I had introduced the SOPM framework right here on PMThink several months ago. I've recently completed a white paper on it, which I'll be making available shortly. SOPM follows a four-step process, UP-IT, which stands for Understand, Prepare, Iterate, and Transform. At each phase, a mindmapping tool like MindManager is extremely useful for brainstorming, facilitation, and visualization. I'll discuss the details in the webinar.

The link to register is below...

Mindjet: Webinars

Einstein Project Management Tip #4: Think Value

And so we continue our series on project management tips from Albert Einstein. Here's another...

"Strive not to be a success, but rather to be of value."

This sums up perfectly the problem with most projects today. They focus on "success" without fully defining what success means. Project managers and PMOs track schedule and budget metrics. Then, at the end of the project, some capture customer satisfaction, almost as an afterthought.

What really needs to happen is to insure value to the customer, and this usually goes way beyond being on time and on budget. We spoke about the need for clear goals. Surely that's part of it. We also need to deliver in small, frequent iterations to provide the quickest value and get more immediate customer feedback.

Customer satisfaction should be measured and tagged as an index throughout the life of a project, just as Earned Value uses indices to track cost and schedule performance. This allows course correction to be made in areas such as goal clarification, communication, and other areas needed to provide good value.

And when the product has been delivered, be sure that the customer can maximize the benefits of the product through proper training, tips & techniques, next steps, or any other items that will help them get the value expected.

These are the very items I've attempted to address with my Service-Oriented Project Management (SOPM) framework, with its four phases of Understand, Prepare, Iterate, and Transform (UP-IT).

More Einstein tips coming soon...

PMO Success Story: A.G. Edwards Case Study

There's an excellent article in CIO Magazine this month showing how A.G. Edwards reinvented its PMO to bring their projects to an 88% success rate (from about 50% originally).

Some key lessons:

• They created a 25-step project management high-level framework of just the high level activities common to all projects. They didn't inflict a detailed application development methodology and left the "how" flexible, as long as the "what" was satisfied. At a more detailed level, they used Primavera for project tracking and dashboard metrics.
  
• They provided leadership training to boost the confidence of their PMs
  
• They moved the project managers from the PMO to the functional areas to encourage collaboration and better align the PMs with the business.
  
• They offered project planning services to assist the distributed project managers with using the new framework effectively (allowing them to use the planning tool of their choice, be it Excel, MS/Word, or a whiteboard). The 25 framework touchpoints, however, are common to all projects for cross-project comparison purposes (I assume enabled in Primavera).
  
• They redefined "success" as "projects that deliver business value." This gives customer satisfaction and business value even greater priority than being on-time and on-budget (note: they still improved their schedule and budget statistics anyway).
  
  This is the essence of the new model and bears repeating. The customer defines success. Under this model, it's quite possible to have a project that is late and over-budget and seen as a raving sucess.
  
• They tirelessly met with stakeholders in individual and group settings to offer the benefits and ask for their support. They used a subtle soft-sell approach with the "bad actors."
  
• They first involved the PMs receptive to new ideas as part of a pilot and them used them to "spread the gospel"
  
• They measured success rates and publicized them in quarterly reports to senior management.

These are all powerful and valid ways to make a PMO successful, and are philosophically aligned with the Service Oriented-Project Management (SOPM) model I've been developing. In this case, these changes collectively served to boost IT's credibility at A.G Edwards significantly.

Here's the full article. Don't miss the sidebar "8 Steps for Improving Project Management."

When Failure Is Not an Option - Editorial - CIO

IT Strategy: Customer Focus SOPM ...

Marc Puich discusses the opportunity for information technology in the biopharmaceutical industry, advocating a simplified enterprise application architecture and a gradual, disciplined approach to operations excellence. I especially like his thoughts on customer focus and feel this spirit should be reflected in the principles of the service-oriented project management methodology that we are developing. SOPM should be customer-centric and its critical path should focus on the essential deliverables for customer success. ...

... "Begin with the customer. Developing an IT strategy should begin with an external focus. This process requires taking a critical look at what functionality is truly necessary to support your customer, versus what would be nice to have. The goal of a system is not to remove people from a process, but to provide the customers with what they need. " ...

Via BioPharm International: Operations Excellence: Perfecting IT Management System Selection for Biopharmaceutical Organizations - Proper application of an IT system can be a critical component to driving efficiency and reducing waste ...

Project Management Holy Grail; Keep Searching

With all the hoopla over the DaVinci Code, I figured I'd share this insightful article from Max Wideman on what many have considered the "Holy Grail" of project management---the ever elusive set of "best practices."

Wideman rightfully argues that any list of best practices is likely tainted, because who'd want to share anything so groundbreaking that it gives them a competitive edge---and if it was so groundbreaking, it would most likely get lost in the survey process that looks for frequently used practices.

So, instead of best practices, we end up with "common practices," which in many cases are extremely flawed.

Better to break new ground and look for unique approaches that fit what you are trying to accomplish, or look for the best examples of what some other organizations are doing. But beware of surveys that claim they show "best practices."

When it comes to project management, you'll just need to keep looking for the Holy Grail. Better yet, try to create one, as I'm attempting to do with my new Service-Oriented Project Management (SOPM)™ model, and others are doing, such as Garry Booker with his Streamlined Earned Value model and his other creations. But you won't see any of these on any "best practice" lists.

Here's the full article...

Max's Musings - The Fallacious Argument of Best Practice - Or the Holy Grail of Project Management

Join the Project Management Revolution; The SOPM Model Takes Shape

OK, I've been fleshing out the Service-Oriented Project Management (SOPM)™ model, and have come up with a more memorable and catchy representation of the four steps, although the actual content is pretty much the same.

The acronym for the four phases is UP-IT (which can symbolize "upping" the level of customer service, saying "up yours" to old ways of doing things, or "upping" the success rates of IT projects---in which case the "it" stands for "IT").

Ready??? Drum roll please......

The four phases are:

• Understand
• Prepare
• Iterate
• Transform

Here's a revision of my previous post on the topic...

1) UNDERSTAND ... Develop an understanding of the problem being addressed, the goals, constraints, the internal environment, the external market, benchmarks, the people and subject matter involved, potential solutions, risks, benefits/justification, and any other knowledge necessary for success. Most of all, understand the customer and what they need to be successful.

2) PREPARE ... After helping the customer obtain approvals if needed, prepare the project organization (resources, roles & responsibilities), operating principles, the infrastructure and tools needed to run the project, organizational alignment, preliminary training needed, communication, and anything else needed for a smooth road ahead.

3) ITERATE... Using the axiom, "Think bold, implement safely," plan, design, build, test and pilot the solution before attempting a full scale implementation. Encourage innovation. Implement in phases to achieve quick wins, earlier benefits, and greater customer satisfaction. Consider iterative prototypes during the design phase. Don't forget additional training needed.

4) TRANSFORM... After each project phase and at the end of the project, evaluate and document lessons learned, customer satisfaction, and benefits achieved (vs expected) for the purpose of transforming yourself and the customer for the better. This includes guiding the customer to help them achieve maximum results with the product or service delivered, and laying the groundwork for their continued success.

Now that I have the framework locked in, I'll complete the model around these four phases. I am absolutely convinced that this model can help increase customer satisfaction and the general success rates of projects.

SOPM; A New Project Management Methodology

Service Oriented Project Management (SOPM) is taking shape as a methodology that fills the gaps in traditional project management, namely a RELENTLESS customer focus and the all-important analysis and benefits evaluation after the project has "completed."

As I fine tune the model, I'll post the iterations here, as a methodology in progress.

The four high-level steps in SOPM are as follows:

1) UNDERSTAND ... Develop an understanding of the problem being addressed, the goals, constraints, the internal environment, the external market, benchmarks, the people and subject matter involved, potential solutions, risks, benefits/justification, and any other knowledge necessary for success. Most of all, understand the customer.

2) ENABLE ... After helping the customer obtain approvals, prepare the project organization (resources, roles & responsibilities), operating principles, the infrastructure and tools needed to run the project, organizational alignment, preliminary training needed, communication, and anything else needed for a smooth road ahead.

3) ITERATE... Plan, design, build, test and pilot the solution before attempting a full scale implementation. Implement in phases to achieve quick wins, earlier benefits, and greater customer satisfaction. Consider iterative prototypes during the design phase. Don't forget additional training needed.

4) EVALUATE... After each project phase and at the end of the project, evaluate and document lessons learned, customer satisfaction, and benefits achieved (vs expected). This includes evaluating how the customer can achieve maximum results with the product of the project, and laying the groundwork for their continued success.

By using an UNDERSTAND, ENABLE, ITERATE, and EVALUATE process, with COMMUNICATE as an overarching activity that extends across all four steps, we adopt a much more holistic and customer-centered approach to project management.

A few key points... Customer satisfaction should be measured at milestones throughout the project, not just at the end. It's as important as monitoring cost and schedule (i.e. Earned Value performance).

Imagine seeing an S-Curve showing Planned Value, Earned Value, Actual Cost, and Customer Satisfaction. Maybe your project is on schedule and on budget, but the customer isn't satisfied with the results (or with the project communication, or a whole host of other issues).

A narrow focus on cost and schedule takes too much of an inward view. Besides, measuring customer satisfaction throughout a project allows for corrective action instead of managing in the rear view mirror.

More to come.

NOTE: I have since revised this model. See my updated entry.

Service Oriented Project Management (SOPM); Bridging Three Worlds

With all this talk about Business Process Reengineering (BPR), and the latest industry focus on innovation, I've been piecing together a model that brings together the best of BPR, Innovation, and Project Management (and even borrows elements of ITIL). I call it Service Oriented Project Management or SOPM. I believe the term has been used, but not in this context, and not as a formal model. I think it's important enough that it needs to be formalized.

There are some that view these three disciplines as separate, or even mutually-exclusive, but they're not. In fact, to be successful, these disciplines need each other. It should go without saying that BPR needs innovation in order to break new ground (resulting in dramatic and radical change, as opposed to incremental change). And project management skills are needed to keep a team on track and manage risk.

Certainly, there are situations where incremental change is quite appropriate, and, for these cases, process "improvement" disciplines such as Six Sigma and TQM are fine. But especially when radical change is needed, we need a superstructure of good project management to lead all phases of a BPR initiative, from the as-is state exploration, through the to-be state development and validation, and to the actual implementation of the initiative.

Likewise, project management in general needs the strong customer focus that BPR brings (usually sorely lacking in most projects). Almost any project can benefit from a BPR-type approach of getting to the root of the customer's problem first-hand, and bringing about dramatic results through innovative thinking. This also takes project management beyond the realm of simple "execution and control".

Using a BPR lifecycle, innovative thinking, and an overall project management approach, we get a holistic methodology that uses the best of each. And, if this is driven by overarching principles from all three disciplines, we can boost our chances of success exponentially.

And finally, there's the customer. EVERYTHING in all of these disciplines must have a relentless focus on the customer. With any initiative, the glue that holds all of this together is a service owner--- someone who understands the customer's needs (and their business) and owns the initiative from cradle to grave (just like an ideal order fulfillment process should be, according to Michael Hammer, the inventor of BPR). Whether or not this should be the project manager is a whole subject in itself, but it should be someone.

If the project manager does assume this role, then they had better have a strong customer and business focus, and be relieved of any project administration duties that aren't adding value to the customer (which can be assigned to a project accountant). In many companies, the project managers may not have the right skills for this role, but that's not to say that shouldn't change.

More to come, as I flesh out and develop the model. Meanwhile, I'm open to your thoughts on this.