What do you think of when someone mentions The National Archives? Possibly, the time-honoured tradition of old government files ceremoniously released to the public between Christmas and New Year. Perhaps, the notion of a storage facility, or an impression of quiet, scholarly research. Given that we have been around in one form or another since 1983, there are lots of possible thoughts. Whatever comes to mind, you should not be surprised to learn that, in the 21st century, we are responsible for so much more.
The National Archives is a non-ministerial department, under the Department for Digital, Culture, Media and Sport, and the official archive and publisher for the UK Government and for England and Wales. Its collection is unique, dating back more than 1,000 years, with records touching on the history of almost every part of the world.
The collection is astonishingly varied, from parchment to websites. Among the 14 million records described in the catalogue are treasures including Domesday, Shakespeare’s will, the confessions of Guy Fawkes, and the last telegram sent from the Titanic.
Domesday manuscript pages
Working with civil servants
We lead the archive sector in England and are an Independent Research Organisation in our own right but, above all, we are keepers of evidence. As its archive, we are the home of the UK Government’s corporate memory, and our records hold insights into some of the most difficult policy issues of the past. Civil servants who consult the records we hold are learning from the direct experience of their predecessors.
We provide a range of services for Civil Service colleagues wishing to consult the records first-hand, at our reading rooms in Kew or remotely at their place of work. Our searchable online catalogue, Discovery, provides details on our collection, and all publicly available records are available to civil servants. Discovery also includes records we hold that are, under Freedom of Information legislation, exempt from public release. Each department works with us through its named Departmental Records Officer, who can advise applicants on the process for accessing these records should they need them for their work.
As a living archive, our collection continues to grow. Last year, we added more than 55,000 new records from across government. The ongoing transition to the 20-year rule for release of government documents means the records being passed to us are increasingly contemporary. This has changed both the public conversation around the government record and sharpened the challenge for us in preserving it, in whatever format.
Refocusing resources
The format of government records has altered dramatically. In December 2018, we released files that revealed how some members of Sir John Major’s government in 1994 did not believe email would ‘catch on’ as a viable communication tool. Today, the record of government is predominantly digital, and that is a major strategic challenge for the archive.
As part of our response to this challenge, we have refreshed our offer to government departments, refocusing resources to better support departments in managing their information, both digital and the large legacy of paper records.
It is not always obvious what to keep, so we are helping departments by providing more targeted expert advice. We support departments’ efforts to maintain compliance with their statutory obligations, and we are leading work to review and update the Code of Practice for record-keeping under Section 46 of the Freedom of Information Act 2000, to reflect the relevance of digital information.
View of the repository in The National Archives
Digital preservation
Becoming a digital archive has been a game-changer for The National Archives. Effectively, we are now running two archives: one tangible (physical papers in boxes on repository shelves), the other intangible (files held in a digital preservation system).
To be a digital archive is to be an institution comfortable with ambiguity and change, one that can adapt rapidly in response to an ever-evolving technology landscape. We are actively engaged with a wide range of challenges posed by the digital government record, both as it currently exists and as it might develop.
Take email, for example – snippets of texts in threaded discussions, with different participants, the conversations forking and sometimes re-merging. Compared to the letters and memos of the pre-digital Civil Service, it is far from obvious where the digital record might begin and end. This is a problem both we and the Better Information for Better Government team in the Cabinet Office are looking to address. We are investigating how we might best use artificial intelligence (AI) to select which emails to keep and which to delete.
Computers are now reasonably good at classification problems, such as distinguishing between personal email and business-related email in a work account. However, classifying emails in ways that rely on their context is a much tougher nut to crack, albeit the technology is advancing very quickly.
Email is now a well-established and mature technology. Meanwhile, we are evolving new ways of communicating, of capturing information and of processing it. Every technology presents new challenges for the digital archivist, in terms of selection, context, preservation and access. How do we know which AI-based deep networks to keep and what do we need to do to preserve them?
Staff at The National Archives photographing historical documents
The risk landscape
There is no long-term solution to digital preservation, nor reliable preservation software solutions that can be guaranteed to function even over the medium term. The digital archive’s risk landscape is, therefore, complex and varied. For example, each type of storage medium (hard disc, tape, etc.) has its own age distribution, which the archive needs to understand. Moreover, the impact on the archive of data corruption depends both on the characteristics of the file format of the records and on the information density of what is being stored.
The National Archives is developing new methods for measuring and managing the risks to the digital archive. We need an approach to risk that is grounded in data, broad enough to encompass a wide range of threats, flexible enough to accommodate changes in our understanding, as well as pragmatic, incremental, explanatory and predictive.
Modelling the risks
We are currently developing an approach, based on what are known as Dynamic Bayesian Networks, to model digital preservation risks. Such networks (probabilistic ones, of cause and effect) are common in robotics, and show promise for a wide range of data-mining applications. Crucially for the archive, they are also iterative, so we can consider the impact of different preservation actions over time in our model.
As technology continually changes, so does the assessment of the risk landscape. We are trying to blend quantitative data with expert judgement in areas where we lack hard data. To do this, we are exploring expert data collection techniques, such as IDEA (Investigate, Discuss, Estimate, Aggregate), to help fill the gaps in quantitative evidence. This involves framing a specific question for experts, who offer an informed view. After discussion and structured challenge, the process is repeated and judgements aggregated to provide a working estimate for the Bayesian model.
Technology systems become obsolete at an extraordinary pace, making it a highly disrupted and disruptive environment. This makes the role of the archivist even more important in sustaining the archive and securing information. Our work involves intervening at the right time in the right way, to mitigate the risks to the digital records we hold. Through development of Bayesian Networks we are at the forefront, internationally, in the creation of new approaches.
Digital services
The National Archives also operates some major digital services, including the government web archive and legislation.gov.uk. We have created a comprehensive archive of European Union law as part of the UK’s preparations for EU Exit. We have also helped to ‘domesticate’ retained direct EU legislation, by adding legislation originating from the EU to the legislation website. We have been busy capturing data about all the changes made by EU Exit Statutory Instruments, so we can produce the UK-applicable versions of the texts for users of legislation.gov.uk.
Archives for everyone
In this article I’ve focused on the archive as a resource for government and a partner in tackling the challenge of digital, but as The National Archives we have a wider historic mission. We believe that the information we hold, in whatever format, is for everyone, because it is about all of us and our shared histories.
An archive needs to be used in order to be useful, and staying relevant to our users is vital.
We are committed to becoming an inclusive archive – tearing down barriers to access and actively reaching out to new audiences. This commitment means we are increasingly becoming a place that people visit to experience and enjoy, as well as to learn.
We are transforming our spaces and engaging people who might never otherwise think of coming to an archive. We have an exciting range of events in our ‘What’s On’ programme – from family days and creative workshops, to themed evening events. These act as signposts or gateways into more detailed, personal research for many people. And this transformation isn’t just at our headquarters in Kew. Our reach online is continuously growing, with 274 million records delivered to online users last year.
A living and growing archive. A global leader in digital thinking. A new kind of cultural and heritage institution – inclusive, entrepreneurial and disruptive, we are redefining what it means to be a 21st-century national archive. And perhaps changing the way people think about archives.
Shakespeare's will
Shakespeare's will
In 2016, The National Archives carried out innovative archival and scientific research into the will of William Shakespeare.
Conservators removed a heavy paper backing and earlier repairs made with silk, to return the 400-year-old document’s appearance closer to its original state. This allowed close analysis of the paper for the first time, using x-ray technology and near infrared light.
The analysis showed that page two of the three-page manuscript was drafted at a different time to the first and last pages. It also revealed significant changes made in both January and March 1616 as the playwright’s – and his family’s – status changed. Initial results suggested that Shakespeare was a canny businessman who revisited his will several times to keep it up to date and secure a financial legacy for his family.
The new research forced scholars to reassess the will, as it cast serious doubt on several accepted theories, including:
• Shakespeare had been ill for some time and had retired to Stratford, where he wrote his will as he lay dying
• he was sour and cold towards his family and left no tender words in his last will and testament
• he distrusted his daughter Judith and her new husband Thomas Quiney, and changed his will to prevent Quiney from benefiting
• he was mean or indifferent to his wife Anne and only left her the ‘second-best bed’ (not the snub it might appear: in Shakespeare’s time, the ‘best bed’ was a symbol of prosperity, reserved for guests, while the ‘second-best bed’ mentioned in the will is likely to have been the Shakespeares’ own marital bed).
Guy Fawkes's confessions
Guido Fawkes's faintly signed first confession
Guy Fawkes is the best-known of the men who planned to blow up King James I during the State Opening of Parliament on 5 November 1605 – the Gunpowder Plot.
James I was a Protestant king, and English Catholics despaired of any return to the old religion. A small group decided to blow
up both King and Parliament with gunpowder and place James’s daughter Elizabethon the throne. They hoped she would marry a Catholic prince and England would once againbe a Catholic country.
The King’s spies discovered the plot. Fawkes was found during the evening of 4 November with 36 barrels of gunpowder in the cellars under the Palace of Westminster, where Parliament was due to meet. He confessed to the plot and named the others involved, signing two confessions – one after torture and another eight days later. The contrast is remarkable. His first signature is weak and shaky where ‘Guido’ can faintly be made out. The second is signed in a steadier hand, ‘Guido Fawkes’.
Fawkes and the other plotters were executed on 30 and 31 January 1606. Ever since, every November, firework displays and bonfires recall the Gunpowder Plot.
John Major's note on Maastricht Bill court action judgment
Part of a message to Prime Minister John Major on the judgment in the court action on the Maastricht Treaty, with the PM's handwritten comments
The Treaty on European Union, better known as the Maastricht Treaty, was signed on 7 February 1992 by the 12 members of the European Community. It founded the European Union and paved the way to the single European currency, the Common Foreign and Security Policy, and cooperation in the fields of justice and home affairs.
In Britain, the ratification process split the Conservative Party in Parliament and Lord William Rees-Mogg went to court against John Major’s government.
Lord Rees-Mogg, worried that ratification might lead to loss of national sovereignty, argued that the process was “legally and constitutionally flawed”, and that the Government had acted illegally by failing to give Parliament a chance to scrutinise parts of the agreement.
The High Court rejected his arguments, with all three judges supporting the Government’s position on the issues. As Rees-Mogg was also asked by the High Court to cover the costs of the legal action, John Major awarded himself a “full gloat”.
The Maastricht Bill passed through the Commons on 23 July 1993, after John Major won a motion of confidence approving the Government’s policy on the Social Chapter.
Recent Comments