Cite this as: Bibby, D. and Ducke, B. 2017 Free and Open Source Software Development in Archaeology. Two interrelated case studies: gvSIG CE and Survey2GIS, Internet Archaeology 43. https://doi.org/10.11141/ia.43.3
Today, computing and software-based research permeate every aspect of archaeology. Even so, the choice of software specifically created for archaeological applications, though slowly growing, is still limited. The potential of Free and Open Source Software (FOSS) still remains to be fully tapped for the benefit of archaeology. In one area, GIS (upon which this article will concentrate), FOSS is readily available for productive use. Database management and statistical software should also be mentioned in this context, and if some open-source software doesn't quite do the trick, then it can easily be modified and enhanced, or even be newly created, as these case studies show (Ducke 2015).
'Free' in 'Free and Open Source Software' can be misleading. It stands for the liberty to use, modify and redistribute software. FOSS is neither 'freeware' nor 'shareware'. Similarly, the term 'public domain' is not applicable, as the latter implies the absence of any form of copyright. The developers of FOSS, however, do not give up their copyright privileges by default. An example of copyright (in this context also called 'copyleft') protection for FOSS is the popular GNU General Public License (GPL). Generally, open-source license terms are very liberal, but there are limits. Some software, for example, may only be used freely for academic and non-commercial purposes.
FOSS is not non- or anti-commercial. It is simply 'open' and not 'closed', as is the case with proprietary software. And it is not primarily created by hobby programmers and geeky enthusiasts. Many FOSS programmers are professionals who contribute open-source code because they get paid or have other relevant motivation, such as developing a software to do a specific task in their own field of interest.
The archaeological community is becoming increasingly aware of the potential of FOSS, but this potential has not yet been fully tapped. As with any sustainable development, the development of FOSS needs long-term financial and organisational security, which, due to a combination of overall lack of money and often short-lived, project-based funding cycles within archaeology, is often not the case. There is also, or has often been, a habitual acceptance of the 'pay-per-license' model. This was the case with the Landesamt für Denkmalpflege Baden-Württemberg (State Cultural Heritage Department, Baden-Württemberg) until the high costs of replacing outdated commercial software, hardware and operating systems used for basic and essential archaeological recording and surveying and changes in proprietary licensing models, forced the hand of the latter institution and led to a rethink that ultimately evolved into a new FOSS initiative – the development of Survey2GIS. In the context of software-based work and research, there is a dangerous trap – and one into which the Landesamt für Denkmalpflege, with respect to archaeological software had, at least to some extent, fallen: research thinking should not be 'application-centric', where the researcher allows the available software to determine the methods. Rather, the thinking should be 'data-centric', recognising that the data represents the most valuable and irreplaceable investment, and that the choice of application software should be made (or new software developed) with data and protocol compatibility and interoperability in mind. With regard to GIS and spatial data infrastructures, a robust technical criterion for the inclusion or exclusion of software is how well it supports the standard protocols and data formats specified by the independent Open Geospatial Consortium (OGC). Besides facilitating the ideas of data sharing and transparent processing, open and standards-based data infrastructures ensure the feasibility of long-term data storage, archiving and accessibility at the same time as alleviating the risk of locking data into undisclosed proprietary formats and thereby possibly losing it forever. From an educational and academic point of view, the data-centric approach is attractive because it favours broader, transferable skills over narrower, application-specific skills. From an economic point of view, it opens up a broader range of investment options and greater control over software spending. As the Landesamt für Denkmalpflege discovered during its own bespoke development of Survey2GIS, it is generally more cost-efficient to modify or, as in this case, even create software that integrates well into an existing, open infrastructure, than to license all the components required for a complete proprietary infrastructure. This was certainly found to be the case for spatial data and GIS, for which complete open-source solutions exist, upon which further development can grow.
GIS is one of today's most important software platforms. It is of critical importance not only to science and research, but also to businesses, consumers, and public agencies. Until recently, however, a FOSS replacement for proprietary desktop GIS, i.e. a system that would not require its users to completely rethink their approach to GIS, that would allow them to continue working without having to convert their data to another format first, and that would cover the entire workflow, from data editing and processing to map publication, was simply not available. The fact that this situation has changed dramatically, and that archaeologists, among others, no longer need to pay for expensive proprietary GIS, is in no small part thanks to QGIS and gvSIG. QGIS is probably the most 'well known' and 'popular' of the FOSS GIS solutions. gvSIG, less well known, is also a powerful GIS – especially the 'CE'-fork, based on the gvSIG OADE (Oxford Archaeology Digital Edition), with much to offer to archaeology. The Landesamt für Denkmalpflege Baden-Württemberg employs both QGIS and gvSIG CE and has been actively involved in the continued development of archaeological tools for and in gvSIG CE (Figures 1 and 2).
The history of gvSIG (Generalitat Valencia Sistema de Información Geográfica) begins in 2003, when the Spanish software developers Iver were awarded a contract by the Regional Ministry of Infrastructure and Transport of Valencia (CIT) to develop a new, functional and stable open-source GIS to be used in cadastral works, spatial planning and the management of public infrastructure, to replace proprietary CAD and GIS solutions for spatial database access. 2009 saw the release of gvSIG 1.9, which was at that time the first FOSS desktop GIS solution that could be considered a fully functional replacement for established proprietary GIS. But productivity was hindered by its 'bugginess', the rough edges of the graphical user interface and the incompleteness of the English translation of commands. After the release of gvSIG 1.9, CIT made the decision to focus all future development on version 2.0, which was to be written from scratch. At that time no release date was set. For the (many) users of gvSIG 1.9, reverting to an older version of gvSIG was not a feasible option, as the version in question, gvSIG 1.1.2, was inferior in terms of its functionality and lacked many capabilities required of a professional GIS solution. Thus, users were trapped between an outdated version, an error-prone version and one that did not yet exist. In the UK, Oxford Archaeology had invested in gvSIG and depended on it as the core element of its FOSS GIS migration. It reacted to the dilemma it now faced by using its own resources to improve gvSIG 1.9 to the point where it could be used for productive work. It was in fact so successful that it finally motivated the CIT's project team to improve upon the code of their version 1.9, releasing version 1.10 (and later 1.11 and 1.12) prior to version 2. gvSIG OADE 2010 ultimately led to the development of a new project: gvSIG Community Edition (CE), supported by a lively consortium of open-source developers. In technical terms, the CE project is a 'fork' of the original gvSIG based upon the Version 1 code base, rather than on that of version 2. At the time of writing, the 64-bit version is vital, well developed and (alongside QGIS) established as an important archaeological, landscape analysis and administrative tool in both the Landesamt für Denkmalpflege Baden-Württemberg and the Bayerisches Landesamt für Denkmalpflege (State Cultural Heritage Department, Bavaria).
Survey2GIS is a light-weight FOSS tool for use in field documentation and surveying, a sustainable program for transferring survey data into GIS. It should be of the greatest interest to the archaeological reader.
The primary motivation for creating Survey2GIS was to gain flexibility and technological independence, as well as the economic aspects discussed above. Faced with an initial situation similar to that of Oxford Archaeology during its dilemma concerning gvSIG, though with a much stronger financial element, the Landesamt für Denkmalpflege Baden-Württemberg needed to find a way to lessen its dependence on costly specialist software for field documentation. This led to an internal review of actual user needs. In 2011/2012, under the direction of the Landesamt für Denkmalpflege's digital archaeology unit, then current field workflows were analysed and individual FOSS solutions were considered. A central element in the Landesamt für Denkmalpflege's strategy was the transition of topographic survey activities from proprietary CAD to (FOSS) GIS (gvSIG CE and QGIS). To make this possible, new software had to be devised to act as the link between the surveying hardware and the GIS.
The result was the development of Survey2GIS, a flexible and user-friendly, cross-platform open-source tool, capable of processing raw survey data and converting the latter into topologically cleaned GIS datasets. Survey2GIS is now a fully developed, compact and flexible solution for handling topographic survey data. It is capable of processing 2D or 3D point measurements into complex geometrical objects (points, lines and polygons), including multipart features and incomplete polygons. The output generated by Survey2GIS is ideal for direct use in GIS. Input data consist of one or more survey data files with coded coordinates (Bibby 2014; Valenta et al. 2015). These data can come from a variety of sources: data collected in the field using, for example, a total station or GPS device, lists of coordinates or even pre-existing cadastral files. Output data is in the form of the widely used ESRI(tm) Shapefile format (2D or 3D), according to geometry type and with complete attribute data. This process can be fully controlled by the user, thus allowing flexible adaptation to individual survey workflows and data structures. The software is user friendly, easy to learn and feature-rich, with detailed procedures to support quality assurance and consistent documentation of all processed data. A simple scripting system allows data-driven, rather than program-driven, input and output. Input and output formats can be adapted to fit the requirements and constraints of virtually any project. During its development, high priority has been given to the generation of topologically correct output, suitable for quantitative analysis in GIS. It was important from the outset that Survey2GIS should not depend on other aids in order to run. Therefore, Survey2GIS is fully functional as a cross-platform, independent stand-alone program (Figure 3). Additionally, it works especially smoothly as a plugin from within gvSIG CE (Figure 4), where the workflow, from input of survey data to the creation and visualisation of attributed vector layers and their visualisation with symbology and labelling, takes place semi-automatically (Figure 5).
At the time of writing, development funding continues to be sustained by the Landesamt für Denkmalpflege. This, however, will not be feasible indefinitely. Possibilities for continuing development funding might be a collaborative platform on the Internet, paid-for support and subscription models, actively advertised at specialist meetings and conventions. One of the most intriguing aspects of the Survey2GIS project is its ability to show how FOSS can unlock innovation potential. Prior to the inception of Survey2GIS, the Landesamt für Denkmalpflege's field workflows had orientated themselves along the lines defined by user interfaces and functionalities of proprietary software, such as various proprietary CAD-systems. With the freedom to create new, customised software, however, also came the freedom to reassess and modify existing workflows in order to make them more efficient. As a result, Survey2GIS is highly customisable and includes a number of features designed to boost productivity in the field. This is a significant return on investment, often overlooked when comparing the license fee savings against the cost of open-source software development and staff training.
This article, while concentrating on the FOSS software gvSIG-CE and Survey2GIS, is by no means a condemnation of either proprietary software or traditional business models. FOSS and proprietary software coexist and will continue to do so for the foreseeable future, as the diversity of user demands and expectations calls for an equal diversity of approaches. However, in the case of proprietary software, excessive or fluctuating license fees, cloud-only services, the risk of vendor lock-in through cryptic data formats and a lack of shared investment options should open the minds of those in research and education to consider alternatives. And as shown by the above case study, at least one large and important authority in German archaeology, echoing a solution initiated by Oxford Archaeology some years earlier, has in the case of Survey2GIS recently put this – possibly groundbreaking approach for the future – into practice.
Portions of sections 1 and 2.1 of this article are based loosely on abbreviated extracts from Ducke, B., 2015.
Bibby, D. 2014 'Survey2GIS: Eine Anwendung für den Transfer von Vermessungsdaten ins GIS'. http://www.landesarchaeologen.de/fileadmin/Dokumente/Dokumente_Kommissionen/Dokumente_Archaeologie-Informationssysteme/Dokumente_FOSS-ARCH/FOSS-ARCH_Bibby.pdf
Ducke, B. 2015 'Free and open source software in commercial and academic archaeology: sustainable investments and reproducible research' in T. Wilson and B. Edwards (eds) Open Source Archaeology: Ethics and Practice, Warsaw: De Gruyter Open. 92-110. https://doi.org/10.1515/9783110440171-008
Valenta, J., Bibby, D., Ducke, B., Canalejo, J. and Schönbuchner, R. 2015 'Survey2GIS: Eine flexible Open-Source-Lösung zum Transfer von Vermsungsdaten in GIS'. http://arch-iv.de/S2G_Workshopagit2015.pdf
Internet Archaeology is an open access journal. Except where otherwise noted, content from this work may be used under the terms of the Creative Commons Attribution 3.0 (CC BY) Unported licence, which permits unrestricted use, distribution, and reproduction in any medium, provided that attribution to the author(s), the title of the work, the Internet Archaeology journal and the relevant URL/DOI are given.