Skip to main content

Crossing the ‘Valley of Death’ – how we can bridge the gap between policy creation and delivery

Posted by: , Posted on: - Categories: Making organisations work well
Tony Meggs, Chief Executive, Infrastructure and Projects Authority

The ‘Valley of Death’ may sound an overly dramatic description for the gap between developing a policy and putting it into effect. However, if you have had the misfortune to drag an ailing policy through that valley, under the beady eyes of circling media vultures, then you will understand. As you will if you’ve been a policy implementer waiting on the other side of the dead zone to try and revive said policy. If you’ve been in either position, or neither, read on to learn how we can bridge that gap.

As Chief Executive of the Infrastructure and Projects Authority, I oversee support for the efficient and effective delivery of all types of infrastructure and major projects. These range from railways, schools, hospitals and housing, to defence, IT and major transformation programmes. We are also there to improve performance over time.


The vast majority of government policies are delivered through the implementation of a project or programme of some description. These projects and programme span a wide range, from capital intensive infrastructure and military equipment projects, through to IT projects and major transformation programmes such as Universal Credit or Courts Reform. Irrespective of their diversity, they have one thing in common: if the projects are not successfully implemented, then the policy objectives are not delivered.

Illustration of the use of video technology in courts reform

In the world of research and development, it is well understood that turning a successful piece of research, a laboratory discovery, into a usable product or service is difficult. That is why the gap between research and successful commercial development is sometimes known as the ‘Valley of Death’, so hard can it be to cross from one side to the other.  In a similar vein, we can think of the space between policy development and policy delivery as another valley of death, because it represents the space in which so many policy initiatives are undermined, sometimes fatally, as they are thrown across the valley from the ‘policy’ team to the ‘delivery’ team.

The six sins of project failure

The common causes of failure in major projects are well rehearsed. They are pretty much the same outside of government as within it, and across a wide range of project types:

  • lack of clarity around project objectives
  • lack of alignment among stakeholders
  • unclear governance and accountability
  • insufficient resources, whether human or financial
  • inexperienced project leadership
  • over-ambitious cost and schedule

Well-designed projects address all these issues, and more, in the vital project initiation phase. Taking time at the beginning to ensure that objectives are crystal clear, stakeholders are aligned, accountabilities are well-defined, and so on, saves enormous pain and heartache later on. Good project initiation maximises the chances of a successful outcome; poor project initiation is a harbinger of failure further down the line.

Initiation phase

It is in this initiation phase that government projects can be at a unique disadvantage. Good project initiation takes time, and time is generally in short supply. The amount available is often determined by the demands of the electoral cycle and the commitment of governments to deliver their agenda within it.

It is entirely appropriate for government to make such commitments and be held to account. For civil servants and policy-makers, these circumstances are a fact of life. If we do the right initiation work, announcements can be made with confidence and promises delivered upon – bridging that yawning gap.

I believe the single most powerful thing that we can do to improve the successful delivery of government priorities is to establish a seamless flow and inter-connectivity between policy conception, policy development, and policy delivery, to ensure that we apply best practice from the earliest phase of policy development.

What does this mean in practice? Here are my top four suggestions:

  1.  Always involve delivery expertise in policy development

Policy teams should continuously consider implementation issues: how will this policy actually get delivered; how can the policy be adjusted to make it easier to deliver; what will it take in terms of resources; how long will it take; what can we learn from similar developments elsewhere?  The most reliable way of ensuring that these issues are addressed up front is by including someone with project or other operational expertise from the very start, as part of the policy development team.

  1. Get an independent assessment of deliverability before announcements and commitments are made

There is always a need for independent expert assessment of project plans. This is to counter the inevitable optimism bias that affects any project in its early phases. The IPA can help here, as can departmental assurance teams, peer review teams from other departments, or external experts. The important thing is to calibrate each project or programme against real world experience from outside the project.

  1. Tailor announcements according to the degree of delivery confidence

There is a delicate balance to be struck between giving out something that’s newsworthy and over-committing on detail. This will give time and space for proper option analysis as part of initiating the project. As the project moves through the appraisal and option selection process, then more detailed and specific announcements can be made.

  1. Understand and embrace uncertainty

Frequently in government we try to define quite specific outcomes, despite having very uncertain circumstances and highly imperfect information. This, especially when combined with optimism bias, can lead to disappointment all around. Policy initiatives, and their associated implementation programmes, often make very broad assumptions as to how the public will react to a policy when it is implemented, and therefore what benefits will be delivered. Unfortunately, the large uncertainty range associated with those assumptions is frequently forgotten as business cases are written and approved, and the benefits banked in advance.

There is a strong case for understanding uncertainty in a more systematic and quantitative manner and trying to define it. For example, if the level of uncertainty is large enough to jeopardise the project, could a pilot project be undertaken to establish an acceptable level?

Universal Credit

Universal Credit – the biggest modernisation of the welfare system in a generation – originally suffered from having delivery timetables and costs committed to and announced before having properly engaged with delivery teams. This resulted in an over-ambitious project that didn’t fully recognise what was required to achieve its transformational aims on the scale envisaged.

This led to a reset in 2013 and a rethink of the entire delivery approach to enable a more realistic prospect of successful delivery. The Department for Work & Pensions (DWP) changed its deadline for complete roll-out within two years to a new ‘test and learn’ approach. This involved rolling out the programme safely and securely in incremental stages, iterating the service as it went, with a multi-disciplinary team in place comprising both Operations and Policy teams embedded in the programme to design and deliver. 

This new approach and the introduction of multi-disciplinary teams was a first for the department, but it has now changed the way we approach large-scale projects into delivery.

These suggestions are all in service of one objective: to increase the probability of successful policy implementation by making delivery planning an integral part of the policy-making process.

The more we can do to operate in a seamless and integrated way from the very beginning, the closer we will come to bridging the Valley of Death and improving the probability of successful delivery of government policy.

For more information about the IPA, visit GOV.UK, email or follow us on Twitter @ipagov.

Sharing and comments

Share this page


  1. Comment by Leila Rushe posted on

    Very informative and gives good understanding of how difficult it is to streamline the new processes within strict guidelines

  2. Comment by Nick posted on

    Probably the best blog I've read here; highly informative, good exemplification, and clear practical signposts to bridging the gap. Great. Worth printing off and keeping IMHO.

  3. Comment by Eric Roberts posted on

    Hi I agree with Leila Rushe things are not as easy as they first look when it comes to delivering certain policies.

  4. Comment by Santosh Sharma posted on

    This is excellent - everyone in a policy role should read this. Will share on LinkedIn