Industry Viewpoint: The Power of community in digital preservation
In recent years, the IFI Irish Film Archive has been getting to grips with the complex challenge of digital preservation. Whilst digital preservation is difficult for all archives, it is fair to say that the size of files to be preserved, speed of technology change and variety of file formats and codecs involved make it perhaps even more challenging for those of us working in the audio-visual sector.
Since 2014, the IFI has been following a Digital Preservation and Access strategy https://ifi.ie/2015/06/digital-preservation-and-access-strategy which is based upon the principles of flexibility and scalability. As a result, we have developed a suite of Open Source tools that support micro-servicing and enable us to fulfil our digital preservation remit within our limited staff and financial resources.
When we secured funding from the State to upgrade our digital infrastructure in 2014, we realised that much of the software and equipment we had purchased was designed for the broadcast sector and had not been created with long-term preservation in mind. Frustrated when a vendor issue wasn’t being addressed, one of our team, Kieran O’Leary, learned to write Python, a computer programming language, and created a script which resolved the problem that was restricting our development. This led to the development of a suite of tools now called IFIscripts which will not only be used by the IFI for years to come but has also proven beneficial to a range of international organisations. These scripts are used daily by IFI staff for activities such as fixity, movement and transcoding. Adopting a micro-services architecture has enabled the IFI to create discrete modular services and applications that can work independently or as part of a larger workflow structure. Being able to create bespoke Open Source tools has transformed the way the IFI works, automating many of the preservation and quality control tasks within our digital workflows.
The IFIscripts were born out of necessity. IFI Archive staff were both restricted by funding limitations and influenced and encouraged by the work of other archives that had adopted Open Source software solutions, particularly the work of Dave Rice at CUNY TV. These organisations shared their tools freely which, in turn, encouraged the IFI to also share our in-house tools. Our engagement with a dynamic community of likeminded archivists and developers has been fundamental to our work. Where possible, we contribute to the Open Source projects that we use, which leads to a sustainable, collaborative engagement with software development and use.
This Open Source methodology has been effective as IFI budgets are limited and we could not afford to employ commercial solutions. Publishing our code on Github, for example, allows for input from the wider digital preservation community. The discovery that we had the potential to solve problems in-house rather than paying large maintenance costs and vendor fees was empowering as we are able to take control of data workflows and reduce the instances of vendor dependency.
We can now create customised applications which address our specific digital preservation needs, unlike those applications being offered by expensive end-to-end commercial software. Most of these scripts incorporate pre-existing free Open Source software, but the scripts are designed so that these tools are tailored to a preservation context. This also allows us to record very detailed log files which document the various preservation events. These log files incorporate the PREMIS data dictionary and follow the Spectrum and OAIS collections management models.
Our tools are freely available on Github for others to use and adapt https://github.com/kieranjol/IFIscripts. We have received feedback from archivists and preservationists all over the world who are using our scripts and have been inspired by our DIY approach. We are delighted to have won two international awards https://ifi.ie/2019/10/ifi-wins-second-international-archive-award/ and https://www.siliconrepublic.com/enterprise/ifi-digital-archiving-ifiscripts for IFIscripts and our contribution to the digital preservation community.
Open Source solutions empower smaller archives with limited resources to preserve their digital collections in a sustainable and flexible way as they reduce dependence on monolithic, expensive, commercial solutions. We hope that our experience in investigating and utilising Open Source tools serves as a positive, innovative model for other archives struggling to preserve their digital collections in a cost effective and sustainable manner.
Kasandra O’Connell would like to thank Kieran O’Leary and Raelene Casey for their assistance with this article.
Wednesday, 22 January 2020 14:41