Project Success is a tricky subject; many c-suite / exec team members simply think that all Projects should be a success so why would they need to define success? This obviously allows ambiguity in the “success” of the project – allowing issues on budgeting, delivery and even profitablility to be (at best) overlooked or (at worst) hidden from the leadership team. In my role as a head of a PMO – Project / Operations Manager, I get asked about the criteria for Project Success a lot , so let’s have a look at this tricky subject.
How to define success criteria
Project success criteria are the standards by which the project will be judged at the end to decide whether or not it has been successful in the eyes of the stakeholders.
Why are project success criteria important?
Organisations are poor at admitting failure, and to help them ignore failure they simply fail to define what failure looks like. We don’t document how we will know if a project has failed – what failure looks like – because thinking about failing is not a good way to motivate the project team when the work has only just started. The absence of a formal definition of failure makes it uncomfortably easy for internal and external stakeholders to brand projects a success (from a false positive perspective) or a failure (when it suits to be negative).
Think about some of the projects that have hit the headlines recently (in any country) Here in the U.K. we have a long history of Project failures.
1822 – Charles Babbage proposed the idea of using J.H. Muller’s concept of ‘a difference machine’ to the Royal Astronomical Society. In 1823, the government gave Babbage £1,700 to start work on the project. But the engine proved to be much more expensive than anticipated, and by the time the government killed off the project in 1842, they had given Babbage over £17,000, without receiving a working engine.
2000 – DSS & Royal Mail Benefit Payment Cards – Setup with the plan to do away with paper payments and to minimise benefit fraud, the Project was cancelled after £1.5bn cost to taxpayers,
2004, the Labour government launched the FiREControl scheme (yes, the use of capital letters within the word are meant to be there). The project was aimed at switching 46 current control rooms into nine regional control centres for fire and rescue services in England, so that the services could respond better to emergencies such as terrorist attacks, floods and rail crashes. But in December 2010, the plan was formally scrapped. In 2011, the public accounts committee said the scheme did not achieve any of its objectives and that eight of the centres were empty “white elephants”, which cost the taxpayer £4m every month to maintain and the entire Project had cost the taxpayer a minimum of £469m of wasted expenditure.
2015 – NHS National Programme for IT cancelled after a £10bn taxpayer spend, )Projects ‘fail’ in the eyes of the media and stakeholders because for people are left to guess what success looks like. Is it delivery on time? Is it delivery on budget? Perhaps those two things really don’t matter much to the stakeholders concerned if they get a great quality result and happy customers.
2016 On Tuesday 7th June 2016, UK Ministry of Defence chiefs told the Commons defence committee on Tuesday that the fleet of 6 new Type 45 destroyers are likely to are likely to suffer engine failure and critical system failures (including propulsion and weapons systems) in warm waters. Rolls-Royce executives said engines installed in the Type 45 destroyers had been built as specified – but that the conditions in the Middle East were not “in line with these specs”.
Yes all the above a UK Government Projects – this is simply because the criteria of success for these projects is agreed upon and monitored by the UK National Audit Office – How many organisations have an equivalent to ensure project success?
Successful organisations take the guesswork out of the Project Success process: they define what success looks like, so they know when they have achieved it. If you want project success, you have to define what success looks like for your project. Perhaps budget is the most important thing to your stakeholders, and quality is taking a back seat on the project. Perhaps customer satisfaction is essential, and you don’t care how many overtime hours (and the hidden cost) the team has to work to get that end result.
Project success criteria are a great tool to use to manage stakeholders and to generate engagement. You can use them to define the project’s goals and track progress – and if your stakeholders stop caring about your success criteria you’ve got an early warning sign that you need to do more to continue to keep them on side.
Project success criteria are the standards by which the project will be judged at the end to decide whether or not it has been successful in the eyes of the stakeholders.
Whatever success looks like, you have to define it so you can own it. There are two types of success criteria that you’ll need to define for your project.
Two types of project success criteria
So you want to know how you’ll know if your project has been a success? You need to identify what success looks like for you and your stakeholders. And the easiest way to do this is to “IStorm” with your team.
During this process you’ll probably come up with success criteria related to the management of the project. These are the success criteria which you can refer to in project audits or the post-project review. They help focus your mind on the ‘business’ of project management and relate to doing the project right. They help you check that you’re hitting all the right targets and are applying project management standards appropriately. Examples would be things like:
- Hold a Project Board / PMO meeting once a month
- Complete project audits in line with the timetable published by the PMO
- Ensure all timesheets are completed by the deadlines
- Achieve 95% compliance on project quality reviews.
Alone these success criteria are not sufficient. They help you measure whether you’re doing a good job but not whether you are actually delivering anything useful for your stakeholders.
Your success criteria analysis should also identify deliverable-based project success criteria which are strongly linked to the business case and the rationale behind doing the project. It’s hard to give sensible examples as they are tied so tightly to what your project is delivering but you should aim for things like:
- Achieve rollout of software to all users
- Train 95% of staff within the two week training period
- Improve customer satisfaction by 65% over the first three months
- Gain Centre of Excellence accreditation for Marketing department.
So, to summarise:
- Project management success criteria: Related to the professional job of running the project e.g. Produce and gain sign off for project initiation document
- Project deliverable success criteria: Related to things delivered as a result of the project e.g. Distribute 6,000 instructional leaflets to households in our target area.
How to document your success criteria
Document your project success criteria in a list. I include the list in the Project Charter or Project Initiation Document so it’s easy to refer to.
Each list item should include:
- Name of success criteria
- How it is going to be measured
- How often it is going to be measured
- Who is responsible for measuring it
You can also capture the output of the measurements here if you want, or move that to another relevant project document. Personally I like to keep them separate, but I don’t see why it wouldn’t work to have your records all in one place.
Of course, you can change them later, add a few more or take some away if the project evolves.
How to measure project success criteria
You have two choices when it comes to measuring project success criteria:
- Discrete: Yes/No
We did or did not do something
Examples: Project delivered on time, company gained XYZ accreditation, new department opened - Continuous: measurable on a scale
We did something to a certain extent, within a target range
Examples: Improve customer satisfaction scores to between 75 per cent and 100 per cent, increase revenue by 8-10 per cent, rebrand 15-20 offices within Quarter Four.
Continuous measurements always include the possibility of being translated into discrete targets. If customer satisfaction was 75 per cent in May, and the target was 60 per cent, you reached the target. If it was 59 per cent, you didn’t. Monitoring benefits on a continuous scale is always better as it allows you to track changes over a period of time. If the customer satisfaction target was reached in May, that’s fantastic. But you cannot tell from a yes/no measurement if it was better or worse than April or what the trend into June is looking like. So go for continuous measurements wherever you can.
Baselining performance
It’s great knowing how you are going to measure success criteria going forwards, but how are you doing today? If you want to capture trending information then you have to take a baseline of current performance as soon as you can, preferably as soon as you have set the success criteria. The problem is that at the beginning of a project there’s normally so much going on that baselining current performance takes a back seat to doing improvements and delivering change.
Still, make time for it, or you’ll have a harder job later working out whether your project has made a difference. It’s great knowing that you are now calling back customers within 20 minutes, but if you don’t know what the call back time was before project was implemented you may very well have made the situation worse – you just can’t tell.
A performance baseline lets you identify the differences in performance in the post-project world related to the things you consider important measures – your project success criteria. Use the same calculations and tracking method to work out your baseline performance as you intend to do for measuring your success criteria later. Otherwise you are introducing even more variables into the mix – keep it simple.
When do I track success criteria?
You’ll take an initial performance benchmark as soon as you can in the project, as we saw above. Then you have to work out how often you want to measure your project success criteria. Each success criteria will have different requirements. Some you can track once a month, others you’ll only measure once more. Some you won’t track very often and then as soon as you hit delivery you might be measuring them daily (like daily quality targets or call handling times).
The true business benefits, on the other hand, hopefully last for a lot longer. Even a one-off project like changing all the office light bulbs to energy efficient ones has durable benefits. The success criteria could be: ‘maintain electricity savings at 40 per cent of previous expenditure for three years.’
The measurement of these over time should be handed to the operations team as part of the project handover when the project closes. You should not keep the responsibility for tracking success criteria (i.e. project benefits) over time. While the project is running, report on your progress so you celebrate the successes as you go. And if there aren’t successes to celebrate, so you can adjust your project approach so that you do hit your success criteria.
Project success criteria: the final summary
To summarise:
- You must define what success looks like for your project or you won’t know if you have achieved it and the danger is that the mistakes will be repeated!
- Success criteria measure what’s important to your stakeholders.
- Document success criteria and get everyone to agree to them.
- Use continuous measurements where possible.
- Baseline today’s performance so you know where you are starting from.
- Track as appropriate and report on your progress.
