Blog post by University of Manchester project developer Tom Higgins:

Yesterday I gave a short presentation on the Data Vault project at an event in Lancaster:

https://www.eventbrite.co.uk/e/research-data-management-solutions-for-your-needs-tickets-17100593335

I based this on the original pitch with a few updates reflecting the work we’ve done over the last couple of months.

Here’s some of the feedback and questions from the event – I think a lot of these are more relevant for “phase 2 and beyond” than the current prototyping:

• How does the Data Vault differ from iRODS? Perhaps the policy model from iRODS could be useful or iRODS could serve as a back-end. There was a comment that iRODS may be more useful where the researcher’s workflow is known and can be encoded into the system (e.g. it’s deeply involved in the day-to-day active data).
• Archivematica (being explored by a project in York) can handle many preservation activities but has a specialist user interface which is not suitable for researchers to use directly. Perhaps a Data Vault could be used to ingest data and hand it over the Archivematica for preservation.
• How would a Data Vault handle sensitive data? Would it be need to be certified? What if the “back-end” was using a certified storage system – would that ease the burden at all? I mentioned that perhaps both a “general” and a locked-down “sensitive” instance of the software could be run in parallel.
• How could a Data Vault handle a dataset that is changing over time? Perhaps snapshots could be captured periodically – would this use a lot of storage space?
• Could data be ingested from instruments automatically? I think this is an interesting one because the researcher will presumably want to access the data on active storage too (e.g. just ingesting into the vault isn’t particularly useful since you’d then need to pull it back out to actually work with the data, but you may want to have a frozen copy of the raw data too).
• How could a Data Vault handle complex data e.g. from a database or an object store? In the simple case a user could export their data (e.g. in a backup format) and store that data (similar to how they might back up a database to a USB drive). Does it make sense for the a vault to try to understand complex data?

Here are some examples of “Active” and “Archive” systems which might be useful targets for integration:

• Box
• Hitachi Content Platform
• DuraCloud
• iRODS
• Archivematica

The development model we chose for the Data Vault is to get us all in a room (Robin, Tom, Claire, Mary, Stuart) and to collaboratively develop the proof of concept system over a few days.  We were kindly hosted by the University of Manchester IT services in their Sackville Street building.

We started by looking at the skeleton framework that Tom and Robin had worked on, and then assigned areas of code to each person to write.  For example work was required on the user interface that the user sees, the broker in the middle that manages the system, and the backend workers that perform the archiving.

All of the code is stored openly in github, and is open source with an MIT license:

• https://github.com/DataVault/datavault

Work is now continuing following the hackathon to complete a few areas of remaining code before the next Jisc Data Spring programme meeting where we can share the system with others.

On 10th of June, the Data Library team ran two workshops in association with the EU Horizon 2020 project, FOSTER (Facilitate Open Science Training for European Research), and the Scottish Graduate School of Social Science.

The aim of the morning workshop, “Good practice in data management & data sharing with social research,” was to provide new entrants into the Scottish Graduate School of Social Science with a grounding in research data management using our online interactive training resource MANTRA, which covers good practice in data management and issues associated with data sharing.

The morning started with a brief presentation by Robin Rice on ‘open science’ and its meaning for the social sciences. Pauline Ward then demonstrated the importance of data management plans to ensure work is safeguarded and that data sharing is made possible. I introduced MANTRA briefly, and then Laine Ruus assigned different MANTRA units to participants and asked them to briefly go through the units and extract one or two key messages and report back to the rest of the group. After the coffee break we had another presentation on ethics, informed consent and the barriers for sharing, and we finished the morning session with a ‘Do’s and Dont’s exercise where we asked participants to write in post-it notes the things they remembered, the things they were taking with them from the workshop: green for things they should DO, and pink for those they should NOT. Here are some of the points the learners posted:

DO
– consider your usernames & passwords
– read the Data Protection Act
– check funder/institution regulations/policies
– obtain informed consent
– design a clear consent form
– give participants info about the research
– inform participants of how we will manage data
– confidentiality
– label your data with enough info to retrieve it in future
– develop a data management plan
– follow the certain policies when you re-use dataset[s] created by others
– have a clear data storage plan
– think about how & how long you will store your data
– store data in at least 3 places, in at least 2 separate locations
– backup!
– consider how/where you back up your data
– delete or archive old versions
– data preservation
– keep your data safe and secure with the help of facilities of fund bodies or university
– think about sharing
– consider sharing at all stages. Think about who will use my data next
– share data (responsibly)

DON’T
– unclear informed consent
– a sense of forcing participants to be part of research
– do not store sensitive information unless necessary
– don’t staple consent forms to de-identified data records/store them together
– take information security for granted
– assume all software will be able to handle your data
– don’t assume you will remember stuff. Document your data
– assume people understand
– disclose participants’ identity
– leave computer on
– share confidential data
– leave your laptop on the bus!
– leave your laptop on the train!
– leave your files on a train!
– don’t forget it is not just my data, it is public data
– forget to future proof

Our message was that open science will thrive when researchers:

• organise and version their data files effectively,
• provide comprehensive and sufficient documentation for others to understand and replicate results and thus cite the source properly
• know how to store and transport your data safely and securely (ensuring backup and encryption)
• understand legal and ethical requirements for managing data about human subjects
• Recognise the importance of good research data management practice in your own context

The afternoon workshop on “Overcoming obstacles to sharing data about human subjects” built on one of the main themes introduced in the morning, with a large overlap of attendees. The ethical and regulatory issues in this area can appear daunting. However, data created from research with human subjects are valuable, and therefore are worth sharing for all the same reasons as other research data (impact, transparency, validation etc). So it was heartening to find ourselves working with a group of mostly new PhD students, keen to find ways to anonymise, aggregate, or otherwise transform their data appropriately to allow sharing.

Robin Rice introduced the Data Protection Act, as it relates to research with human subjects, and ethical considerations. Naturally, we directed our participants to MANTRA, which has detailed information on the ethical and practical issues, with specific modules on “Data protection, rights & access” and “Sharing, preservation & licensing”. Of course not all data are suitable for sharing, and there are risks to be considered.

In many cases, data can be anonymised effectively, to allow the data to be shared. Richard Welpton from the UK Data Archive shared practical information on anonymisation approaches and tools for ‘statistical disclosure control’, recommending sdcMicroGUI (a graphical interface for carrying out anonymisation techniques, which is an R package, but should require no knowledge of the R language).

Finally Dr Niamh Moore from University of Edinburgh shared her experiences of sharing qualitative data. She spoke about the need to respect the wishes of subjects, her research gathering oral history, and the enthusiasm of many of her human subjects to be named in her research outputs, in a sense to own their own story, their own words.

Links:

• Scottish Graduate School of Social Science
• FOSTER Open Science
• Research Data Management Training – MANTRA course

Rocio von Jungenfeld & Pauline Ward
EDINA and Data Library

Meeting our Needs

Since the early 2000s we have been looking for suitable software to manage our archives in a holistic manner. We began to deliver online catalogues at this time via various project initiatives, with metadata encoded as EAD/xml, but this only dealt with resource discovery and was quite cumbersome. Moreover, along with other digital developments, the work inhabited one of a number of parallel silos.

As time moved on, we got better at developing systems to move different elements of work from the analogue to the digital but were still some way off developing or finding a comprehensive, robust and sustainable way to join things up in a meaningful way. This changed when we began to investigate Archivists’ Toolkit in 2011. Although we had looked at it in one of its earlier versions, we were surprised to see how much subsequent developments had brought it quite close to ticking everything on our wish list. It was lacking a resource discovery layer but a successor product, ArchivesSpace, was already planned and would include this.

From Archivists’ Toolkit to ArchivesSpace

We therefore began looking at Archivists’ Toolkit in more detail, assessing issues such as functionality and usability but also those of sustainability and interoperability. It scored very highly, high enough for us to be able to make the business case to commit to ArchivesSpace and obtain the internal funding to sign up as Members.

The involvement of the profession in the development of ArchivesSpace has been and continues to be crucial. What has been developed is not just other people’s idea of what the product needs to be but what we as archivists actually require. Although heavily influenced by the predominant US partners and the specifics of US practice, it has been developed in way that is equally intelligible to others and easily customisable to reflect local needs and terminology.

Priorities and Impact

We originally focused on moving our behind-the-scenes work over but then switched to frontloading our resource discovery, migrating existing EAD xml files and also retro-converting a wide range of old spreadsheets, databases and similar. In terms of impact, this both provides evidence that our business case was sound but, most importantly, meets growing user expectations of what and online catalogue should deliver.

Phase one saw the delivery of nearly 17,000 catalogue records along with over 22,000 authority terms. We still have more to add, along with a whole range of management metadata about accessioning, locations etc. This will feature in Phase 2.

Because the source metadata has been drawn from a variety of legacy sources, there are issues of consistency and quality to be addressed. These are outstanding issues which could never be solved just by getting the metadata into ArchivesSpace. However, with all the metadata now in one place we can now look to quantify and rectify them. Experience told us that’s users would often rather have partial metadata rather than no metadata at all so we chose to go for a warts and all approach, only correcting what was obviously erroneous at this stage.

Community and Participation

We are proud to have signed up as the first European partner and the support we have had from a growing community of ArchivesSpace users and developers. This discussion is also two-way, with us feeding ideas back for future development.

Locally we are also more fully integrated into developing solutions that deliver all our collections online, through a suite of applications and interface that work together, improving user experience and improving how we manage the collections themselves.

Next Steps

We still have lots to do with the system to leverage the full functionality of the system and fully showcase our amazing archives collection. So watch this space.

View the online catalogue.

Read about this from a technical perspective