1. Framework

In England, there is no legal requirement to deposit physical or digital archaeological archives, no licensing of archaeological practitioners, and no statutory requirement for the existence of archaeological repositories. Activity is guided by professional standards and ethics, influenced by the Chartered Institute for Archaeologists (CIfA), and the requirements of planning archaeologists and archive repositories. While there may be an obvious benefit to contractors in depositing physical archives, including reducing storage costs and meeting planning requirements, as well as demonstrating mature working practices, the benefits of archiving digital data remain abstract. They include public benefit, meeting professional responsibility, and maintaining reputation but digital skills remain undeveloped.

In 2019, the Association of Local Government Archaeological Officers (ALGAO) commissioned Landward Research to produce their review Archaeology in Development Management [PDF] (Rocks-Macqueen and Lewis 2019), which explores the impact of the existing system, presenting figures for the volume of archaeology being undertaken.

The review found that the majority of archaeology undertaken is developer-funded (commercial) and estimates the number of archaeological investigations for England and Scotland to be 10,000. If the ratio of predetermination recommendations for the separate countries can be applied then 88% of those are in England.

The National Planning Policy Framework [PDF], the government's planning guidance, asks that

'local planning authorities should require developers to record and advance understanding of the significance of any heritage assets to be lost (wholly or in part) in a manner proportionate to their importance and the impact and to make this evidence (and any archive generated) publicly accessible.' (paragraph 199)

This is supported by footnote 64:

'Copies of evidence should be deposited with the relevant historic environment record, and any archives with a local museum or other public depository.'

2. Investigation Event Recording Infrastructure

The high volume of archaeological investigations undertaken each year in England requires a robust and efficient system to ensure these events are recorded; this information also acts as the high-level metadata that enables archives to be findable.

The investigations are recorded in non-statutory Historic Environment Records (HERs) or similar. The main method of updating these is through OASIS, an online system for recording archaeological activity where digital copies of project reports and supporting images can be uploaded and found through the Heritage Gateway.

A current review of OASIS aims to enhance monitoring of the archiving process, improve data standardisation and increase the numbers of HERs using OASIS.

3. Repositories

The repositories for physical archives are mainly museums, mostly run by local authorities and recent surveys by the Society for Museum Archaeology (SMA) show that both coverage and archaeological expertise is declining. The SMA also reported that some museums acknowledged they did not have the resources to curate digital archives, but others collected digital data without understanding the implications, responding with a focus on data media (disks) and not digital preservation strategies, file format requirements, and metadata (Boyle et al. 2018; SMA 2020).

In 2017, the government commissioned the Mendoza Review into museums in England. This recognised a need to improve the long-term sustainability of archives generated by developer-funded projects and requested Historic England make recommendations to the government to resolve this issue (Recommendation 27; Mendoza 2017, 47). Those recommendations included 'reliev[ing] museums of the expectation that they should attempt to curate digital archive material from archaeological projects, in favour of their deposition in a Trusted Digital Repository that will guarantee the preservation and accessibility of digital material, such as the Archaeological [sic] Data Service' (https://historicengland.org.uk/content/docs/consultations/he-response-to-dcms-mendoza-review-mar18-pdf/).

In the UK, the Archaeology Data Service is the only trusted digital repository that specialises in heritage data, and thus on implementation of the recommendation, England will go from limited collecting of digital data to full geographical coverage, not yet achieved for physical archives.

4. Digital Archiving Projects

4.1 Planning for archives

In 2019, ALGAO, in response to the Mendoza recommendations, reviewed the relationship between planning policy and archaeological archives in Planning for Archives: Opportunities and Omissions (Donnelly-Symes 2019).

The project explored the relationship between national planning policy (as applied locally) and the creation and management of archaeological archives, examining how the policy was enacted and its relationship with 'Archaeological Resources in Cultural Heritage: A European Standard' (ARCHES) (Perrin et al. 2014, https://www.europae-archaeologiae-consilium.org/eac-guidlines). The resulting report summarised the findings of an ALGAO survey and produced a set of final recommendations, finding that planning guidance required definition of the terminology it used including 'archaeological archive', 'digital archaeological archive', and 'publicly accessible' and that supporting planning guidance should have greater detail on how archiving should be achieved, and where data should be archived.

It also found that although there was variation in how guidance was applied (see Table 1), the majority of respondents did expect most aspects of archiving to be dealt with at some point within the archaeological planning process.

Table 1: Percentage of ALGAO:England respondents that indicated at what stage their authority refers or expects reference to, a particular aspect of archiving

	Pre- application	Brief or specification	Written scheme of investigation	Post-excavation assessment	Updated project design	Final or archive report	Archive preparation	Do not refer	Other
Identification of a repository	24%	53%	94%	43%	53%	45%	29%	0%	2%
Acknowledgement of potential costs	18%	25%	57%	27%	27%	25%	24%	24%	20%
Transfer of title	2%	20%	51%	31%	29%	31%	45%	27%	8%
Use of potential selection strategies	10%	18%	65%	53%	41%	25%	35%	12%	10%
Any unique requirements	16%	22%	55%	65%	53%	49%	33%	4%	16%
Estimation of size of archives	6%	8%	33%	59%	41%	27%	31%	12%	12%
Other claims to material	14%	29%	75%	51%	41%	37%	29%	6%	10%

The EAC definition of an archaeological archive and supporting principles were also reviewed to determine whether they were reflected and achievable within national planning policy. This identified incomplete coverage of repositories, not all repositories providing deposition standards for digital data, and poor planning such as under-budgeting for deposition, leading to failure to meet fully the EAC standard across England.

The review found that a lack of effective archiving of digital material was a significant issue and that compared to physical archives, digital data was given substantially less attention from the sector as a whole; for example only 49% (25/51) of planning archaeologists expect all digital archives to be deposited with an accredited digital repository, while 23.5% (12/51) of respondents stated that they have no requirements for digital archiving (Donnelly-Symes 2019, 22). Therefore, a significant amount of work is undertaken in areas where there is no expectation from the local authority for data to be deposited with an accredited digital repository.

ALGAO:England, working with the ADS, gathered some concerning figures for deposition rates: comparison of the number of archives deposited with the ADS against an estimated number of projects found that, at best, 2-3% of all commercial projects have been digitally archived with the ADS. This is corroborated by a comparison between 2017/18 ADS deposition rates and numbers given in Archaeology in Development Management (245 deposits and 2200 predetermination investigations/8000 investigations) (Rocks-Macqueen and Lewis 2019). The 2017/18 financial year saw a dramatic rise in digital archives released using ADSEasy from 80 to 245 but the majority were from only three authority areas. These figures highlight how influential and necessary planning requirements and monitoring are to the deposition of digital data.

For solutions to this ALGAO:England recommend:

• Contracting units should be expected to state the accredited digital repository where the archive will be deposited within proposal documents.
• Contracting units should also be expected to provide a Data Management Plan (DMP) from the outset of each project and requested proposal documents.
• The current ALGAO:England 'Advice Note for Post-Excavation Assessment, October 2015' should be updated to refer to the need for PXAs to include information about the accredited digital repository where the digital material will be archived (p.24).

4.2 Work digital: think archive

In 2018, Digventures with the Chartered Institute of Archaeology were commissioned by Historic England to create a sector standard and guidance for managing digital archaeological data. This included a definition of a digital archaeological archive, guidance for the integration of data management into normal practice, clarification of roles and responsibilities for data management and recommendations for updates to relevant CIfA standards (https://www.archaeologists.net/digdigital). In 2020 the project is in its second phase including training, dissemination and promotion.

Initial work included a survey, aimed at finding out how archaeologists think about, plan for and manage digital data on a daily basis. The 185 respondents included a mix of data creators, repository curators and planning archaeologists across the UK.

Respondents generally considered their knowledge of standards to be fair to good (question 3.1), but there was less use of non-sector-specific standards and guidance. Table 2 summarises the answers to question 5.4 'In your experience, to what extent is digital data actively managed during project delivery in the following ways'. The contradictory figures for the production of metadata tables (row 5) and whether the data are archived (row 8) undermine the positive response to knowledge (Q3.1). If we consider metadata to be a basic requirement for digital archiving then an archive is incomplete without it. The sector's understanding of digital archiving is shown to be immature, which undermines the future usability of data. Data 'archived' without metadata are deposited with physical archive repositories that also accept digital material. In the survey, 46% of 24 repositories required digital data be deposited with ADS or similar, 46% required digital data to be deposited as part of the physical archive and 8% required digital material to be deposited with them and with ADS. This, combined with a lack of requirements for metadata, creates a variation in the costs in archiving that means it is cheaper not to deposit at a Trusted Digital Repository, which is likely to undercut those practitioners that work to professional standards. Seven of those repositories that collect digital material said they were unable to make digital data accessible to researchers, which means that they are unable to fulfil their duty as a repository.

Table 2: Summary of question 5.4: 'In your experience, to what extent is digital data actively managed during project delivery in the following ways', adapted from Parker Wooding and Forster 2018, table 3

	All projects	Some projects	Rarely	Not at all
DMP	33	34	41	23
OASIS	77	36	4	16
Metadata planning	29	51	31	18
Filename conventions	61	40	24	12
Selection	39	51	28	14
Metadata tables	17	47	38	30
Accessible data	10	56	34	31
Archived	60	56	10	7

The resulting guidance will be available as part of an online toolkit. Its key elements include working to a Data Management Plan (DMP), deposition with a Trusted Digital Repository, and following the FAIR principles (findable, accessible, interoperable and reusable). This, together with ALGAO's recommendations, and the SMA's newly updated Collections care guidance (http://socmusarch.org.uk/training/smart-project/), address the recommendation to relieve museums of the responsibility for digital archaeological archives while providing a level playing field for costing and managing archaeological projects.

5. Discussion

Currently in England deposition rates of digital archaeological material are unacceptably low, despite 20 years of data generation, increasingly in born-digital form. The COVID-19 pandemic will likely be a driver for change; its impact on working patterns will drive both the digitisation of physical data and the creation of born-digital data to enable flexible working. Recent projects detailed above will provide the first foundations to improving the quality and deposition of digital data, and have identified that:

• The terminology used in current archive guidance can lack definition and leaves 'room for manoeuvre'
• There is a confusion over what constitutes a digital archaeological archive, leading to poor standards of metadata creation and the misconception that a digital report is equivalent to an archive
• There is a lack of standardisation of how data are treated in the planning process
• Both planning archaeologists and repositories of the physical archive do not always specify deposition of the digital archive
• Data creators and repositories for the physical archive are unable to produce 'publicly accessible' digital archives, as defined by the ARCHES standard
• Planning archaeologists, HERs, curators etc, struggle to monitor the creation, management and deposition of digital archaeological archives
• Data creators, although aware of sector-based guidance, have less familiarity with non-sector specific guidance.

Although unquantifiable, there are likely other barriers to digital archiving, such as perceived costs and inexperience in predicting archive volume, resulting in little understanding of financial implications, protectionism over data and data standards, conceiving the commercial value of data taking precedence over public value and retrospective application of data management processes, so that deposition requirements cannot be met and/or costs are disproportionate. Because digital archiving has not been fully embraced for common file formats, the sector has not moved on to considering the archiving of specialist data.

These recent projects are providing solutions, including:

• Definition of a digital archaeological archive
• Ensuring the deposition of digital archives is covered by standards and guidance, including planning law
• A requirement that digital archives are deposited with a Trusted Digital Repository
• Recommending professional bodies adopt standards and guidance for digital archives, and include this in accreditation schemes
• Within standards and guidance, ensure that terminology is well defined, in particular; 'archaeological archive', 'digital archaeological archive', and 'publicly accessible'
• Creation of sector-specific data management guidance built on best practice within the digital curation sector
• Enhancement of event recording systems to allow monitoring and communication of the archaeological archive process
• A requirement for data management to be guided by a data management plan (DMP) that includes selection and preservation
• A supporting toolkit that includes:
  • Metadata templates
  • Exemplar DMP, file format, folder structure, file naming conventions
  • Promotes timely metadata creation, and defines roles and responsibilities
• The toolkit will be supported by accessible training.

These measures are intended to ensure there are no gaps in process and standards and, although sobering, this is clearly needed when deposition rates remain low. Continuing work is needed furthermore, to support digital archiving and the application of FAIR principles; including continuing to develop data standards and thesauri, support and training for Linked Data and ontologies, and researching the archiving of specialist data.

This regulative approach is driven by the need to counteract the effects of commercial funding in archaeology and also by a lack of appreciation for the public value of archaeological archives.

Landward Research's Archaeology in Development Management report (Rocks-Macqueen and Lewis 2019) included the financial benefits of planning archaeology; commercial archaeology contributes £218m to the British economy. The Heritage and the Economy 2019 England report (Historic England 2019) revealed the heritage sector brought a gross added value of £31.0 billion to the economy, with a tourist spend of £17.0 billion each year. Archaeological archives are a product of the planning process and a part of our heritage but their economic and public value is not reflected in their resourcing. The heritage sector needs to advocate for archaeological archives to be resourced appropriately, converting the public value and economic benefit of archaeology into funding for their preservation and accessibility.

Digital archives, developed using the FAIR principles, provide the opportunity to improve the accessibility and reuse of archaeology and increase the public value of archaeology. This will support arguments for funding fairly to reflect the role of the archaeological archive in providing evidence for the past.

Mendoza's acknowledgement of the archiving crisis and the subsequent acceptance by DCMS of Historic England's recommendation that digital archaeological archives be deposited at a Trusted Digital Repository, is both justification and catalyst for change. Continuous effort and advocacy by interested parties have been significant in initiating that change and the development of standards and guidance to implement it. This article was entitled Red Sky at Night because it is a sign the weather will be better tomorrow, in this case, hopefully, it will be FAIR.

Acknowledgements

I would like to thank: ALGAO:England, Society of Museum Archaeology, Landward Research, Charter Institute For Archaeology, Digventures, the Archaeological Archives Forum, Ben Donelly-Symes, Manda Forster, and Duncan H. Brown, for their support when writing this report and the research on which this report is based.