Skip to main content

Open data

Posted by: , Posted on: - Categories: New techniques

Open data will help reform public services, as well as improve accountability and generate economic growth. The Civil Service can reap the benefits if it learns quickly, says Paul Maltby, Director of Open Data and Government Innovation at the Cabinet Office.

The world is in the middle of a data revolution. More and more data is being captured and shared, and sophisticated digital tools for handling it are now commonly available. Whole industries are emerging to make use of all this data, and our lives are changing as a result.

Government has a unique role to play in this process. Funded by the taxpayer, delivering public services and acting on behalf of the country, departments and agencies generate and hold vast amounts of data. Since 2009, the Transparency Team in Cabinet Office has been leading work to open up this data to everyone. It is already being used to hold the Government to account and generate economic growth. But it also has the potential to significantly improve the design and delivery of public services.

What is open data?

Data is open if anyone is free to use, reuse, and redistribute it in any way they like, free of costs and restrictive licences. The Cabinet Office Transparency Team runs, the world’s leading data portal, featuring well over 10,000 datasets.

Open data achieves its full potential when it can be accessed automatically by applications and websites, combined with other data sets, and used to create insights and services. A step-by-step beginners guide is provided on the site, alongside interactive content for more advanced users.

How open data is driving new-tech economic growth

One of the most successful and prolific areas where open data has gone into mass public use has been through the multitude of transport information apps that allow citizens to better plan their journeys. For example, Transport API is the company that lies behind some of the most popular transport timetable smartphone and web applications. Based within Tech City’s part-government funded Open Data Institute they aggregate transport data from a variety of sources and provide this information as service for app developers and organisations, including Transport for London. Going beyond apps, tech- aware companies are using data from their own businesses alongside purchased data and open data to improve their decisions and find a competitive advantage. McKinsey recently estimated the potential of open data to the world economy is in the region of $3 trillion a year. Examples are companies such as Experian QAS and Deloitte Analytics that provide data consultancy services to other companies helping them decide where to open a new store, how to exploit new markets, or better target products and services at consumers.

Changing how Government works

Open data is changing the way civil servants work. Since the launch of uk, direct interaction and collaboration between the civil servants running the site and its users has become a fundamental part of the way the team works. Data users are able to request particular datasets from Government through the team; and they are also able to help each other extract value from the data. Many of the Transparency Team work with business and community users of data through social media channels.

Olivia Burman, Head of Relationship Management in the Transparency Team, tells how this responsiveness applies even to individual users. “A surfer, tired of not being able to use Tregantle beach because it was being used for firing range practice, requested the firing range data from We helped the Ministry of Defence publish the requested data as open data. Now any surfer wanting to use the beach knows when they can surf there.”

This greater level of interaction has huge potential to serve users better, helping get services right more quickly. But it also means the opening up of different types of accountability, sometimes personally-held, and sometimes experienced at junior civil service grades.

Users know our names, know what we do, and tell us when we’ve got it wrong. The Transparency Team is certainly not unique to government in this regard, but it feels like an experience likely to be replicated in other areas of government in months and years to come.

How open data is improving the accountability of government to taxpayers

Citizens have a right to know what the Government is spending their money on, and the impetus for this is only strengthened during a period of austerity. Across the world transparency and open data are helping reduce public sector costs and provide a disinfectant for corruption. The open data agenda has driven the release of spending data which has allowed people to see more of the Government’s accounts through websites such as the Open Knowledge Foundation’s Open Spending and the Government Interrogating Spending Tool (GIST) built by the Government Digital Service. You can see all the legislation that the UK parliament has passed since 1267 at and open data also helps us see how Government is structured, how to get in contact with senior officials, and even how much they are paid.

Open data has proven to be an important ally of public service reform. The opening up of data once restricted to use within the public sector creates opportunities for innovative companies to find new insights and begin to provide complementary products and services. For instance Ben Goldacre and Open Healthcare worked with the analytics firm Mastodon C to analyse millions of lines of historical prescriptions open data. They graphically demonstrated where £200m of savings can be found within the NHS if particular GPs prescribed cheaper generic statin drugs rather than their branded identical alternatives. This appeal to innovative external talent through the provision of open data is also helping officials in the Cabinet Office get a better deal on contracts by encouraging more innovative small and medium- sized enterprises to bid for work using Contracts Finder and G-Cloud.

Open data is being used by public service reformers within government to help prise open bureaucracies and professions, to put more information in the hands of citizens, giving them more choice and a stronger voice in the public services they use. The widespread availability of open data on schools has prompted the Guardian GCSE Schools Guide which helps parents give a preference for the most appropriate schools for their children. Mark Barrett, an NHS official from Leeds who took on a new role in the city to promote the use of open data, produced a GP Ratings app in his spare time. It allowed patients to compare GP surgeries in their local areas, and included star ratings that indicated whether or not patients would recommend the surgery to others, the numbers of male and female GPs, and even indications of how helpful (or not) the receptionist teams are.

Some civil servants are already trying to use the valuable data available to improve the quality of policy or operational work within government. For example, we know of policy makers that use to access complementary datasets from other departments, and we hear about civil servants using the government organograms to find senior people and understand the business structure in other departments. But they are still relatively few and far between.

Perhaps the best opportunity to combine operational effectiveness and public transparency is in the fast-developing Performance Platform run by the Government Digital Service. This is helping service managers see the performance of their organisations better, and make changes in response. During 2014, the Platform will develop fast both in the number of services available, and the depth of data that it presents.

The lack of knowledge is in part due to the IT infrastructure within government. It is also due to the lack of coding skills necessary to use and manipulate the data in the most sophisticated ways. Reform measures have been introduced to fix the former issue and these are being implemented by the newly- formed Technology Leaders Network. And open data is a resource that can provide useful insight at many levels of expertise, even if that just means tracking down a useful table via Excel, or going through the step-by-step beginners guide on uk to learn how to visualise basic departmental data in a map. Policy makers in the civil service in the near future will routinely make use of data in this way to understand their subject and to explore alternative policy ideas.

The Transparency Team is trying to do something about this, according to Olivia Burman. “We help departments get open data out to the public. We also help departments understand the opportunities they have for using open data – and not just data specialists, but anyone willing to listen. Recently at the Ministry of Justice we talked to 60 people at an all-staff event. But that isn't enough, so from this month we will be spending more time embedded in departments, understanding their challenges and exploring how open data can benefit them.”

The UK currently leads the world on open data according to the Open Knowledge Foundation and the Web Foundation, both respected independent campaign groups. But we need to ensure that the transformation does not stop at Government’s door. More civil servants must step up and become creators with data, not just producers of data or passive consumers. And more of us need to embrace the levels of collaboration with citizens and expert communities that are are now possible.

If you want to know more about how you can use open data, contact me on Twitter via @_OpenP or email the Transparency Team here.

Don’t forget to sign up for email alerts from CSQ

Other CSQ articles you may be interested in:
Opening up the energy debate
Putting data to good use: helping offenders find work

Sharing and comments

Share this page