New feature in Edinburgh DataShare: the REST API

Ever wanted to get a table of the details of all the datasets on DataShare to do with Scottish history? Or matching some other criteria, possibly on specified fields? If so, the new API (Application Programming Interface) can help.

DataShare now has a REST API, which you can use to query our metadata. An API makes the database’s contents accessible for requests from external servers, through a command-line, which allows external users to script such requests. The DSpace API also provides its own web-based query client and report client. These pages allow users to use a graphical interface to quickly build a query and see the results in a table, all in the browser.

The DataShare REST API page starts with a link to our plain-English explanation of how the API can be used:

Edinburgh DataShare DSpace REST API 

We would like to hear from anyone who wants to use the API. Please try it out and let us know what you find useful! Email us at data-support@ed.ac.uk .

Examples using the graphical query builder

I wanted to find datasets where I could add a link to the associated publication. This is a bit of a challenge for us, since users typically deposit their data with us under embargo before the associated paper has been published, and we do not have an automatic way to detect when or whether an associated publication has appeared.  I used the query builder to find the IsReferencedBy value for deposits accessioned in 2017. The plain-English guide on the wiki provides the steps I went through to do so:

How to use the DataShare REST API 

This feature may be of use to colleagues who support organisational units at University of Edinburgh which don’t align precisely with the Collections structure of DataShare – the API lets you query on multiple collections through the reporting tool. We’d love for colleagues to contact us if their teams have published a new paper containing a data citation of their DataShare deposit, so we can add the details of the publication to the DataShare Item’s metadata, resulting in a hyperlink appearing on the dataset landing page.

I wanted to find datasets with an embargo date in December. This is a challenge for us because users often set their embargo expiry date to Hogmanay, which means their one-week reminder would arrive on Christmas Day right in the middle of the university’s winter break. But many other fields contain dates with December in them, so it has not been practical for me to search for this using the graphical interface. So I used the API to search specifically in the dc.date.embargo field. See the screenshot below. The API helped me find the datasets whose embargo date we needed to extend, or else lift the embargo outright, allowing us to contact the depositors in good time to ask them whether a paper had been published or more time was needed.

Screenshot of the output of the REST API

Results showing datasets with an embargo date in December 2021

Thirdly, to demonstrate the power of this tool relative to the non-specific Search I chose a topic with very common words to show how to use the query builder to focus in on results avoiding spurious matches.

Using the existing ‘Search’ function on the homepage I searched for ‘history Scotland’. This produced 39 matches, some of which have nothing to do with historical research or Scotland, but merely mention a funder “NHS Research Scotland”, and mention the history of the research field in passing to provide a little context. Most of the matches are interesting, but some are not relevant.

Whereas when I set the API query builder to search for ‘history’ in the research area (subject classification), and ‘Scotland’ in the field for geographical metadata ie dc.coverage.spatial. This provided me with a short list of high quality matches, three datasets of historical research to do with Scotland – see the screenshot. This is a useful tool for narrowing a search.

Screenshot showing the input, and the output on the API query builder webpage

A search for two very common words in specific fields produces high quality results

Enabling the API

The REST API is a feature of the underlying DSpace repository software. Our sysadmins configured the API with great care to block certain commands and enable only the ‘GET’ commands that are needed for appropriate queries using DSpace config settings (further info DSpace 6 Documentation on the Lyrasis wiki ).

The Future

In the international DSpace repository community, we’re aware the API is used for integration with at least one CRIS (Current Research Information System) and quality tool applications (Andrea Bollini, 4Science, private communication). We understand the API of the newer DSpace 7 contains significant changes compared to that of DSpace 6, which we’re using for Edinburgh DataShare.

We’re aware of only a few examples of the API being used by individuals for occasional metadata queries. But we will watch with interest to see how the DSpace 7 API will be used.

 

Pauline Ward
Research Data Support Assistant
Library and University Collections
University of Edinburgh

Diverse climate change data in DataShare’s newest thematic collection

‘Code red for humanity’ was the galvanising message of the sixth report of the Intergovernmental Panel on Climate Change published on 9 August. The report draws on thousands of academic research projects. Research data is vital to understanding the nature and scale of the challenge of climate change, and the necessary deployment and application of solutions.

We decided to draw together the datasets relating to climate change to showcase them on a single Collection page on Edinburgh DataShare, our research data repository. In part this was prompted by a new deposit from Oliver Escobar in the School of Social and Political Science – data from citizens’ assemblies debating wind farms. Our DSpace repository allows us to ‘map’ an Item to Collections other than the one to which it belongs, resulting in the dataset being listed in more than one Collection. Edinburgh DataShare contains a wealth of research datasets from an extremely diverse array of academic disciplines, reflecting the strengths of the University of Edinburgh, and so it is with our climate change research:

Climate Change Collection

We added many datasets from our School of Geosciences: one dataset from Ian Goddard and Professor Simon Tett demonstrated how urbanisation has affected temperatures in the UK, and includes a map showing heat islands around our major cities. Professor Tett said:

“To truly understand how climate change might impact society we need to bring together many datasets developed by many researchers so that other researchers can use them for their own studies. DataShare enables this.”

Goddard, Ian; Tett, Simon. (2018). “Software and data used in the study ‘How much has urbanisation affected temperatures in the United Kingdom'”, 1990-2017 [software]. University of Edinburgh. https://doi.org/10.7488/ds/2370.

Climatological data and toolkits for public engagement around climate and natural resources came from Professor Marc Metzger – including various kinds of maps, a board game and posters showing natural resources.

One dataset was a description of an artwork, a quilt representing global temperature measurements. Posters on the wall show the years, so as to provide a time axis for the temperature data represented in the colours of the patches in the quilt:

Photo of quilt hanging on the wall of an exhibition space

World temperature quilt on display at the Data-X exhibition

Zaenker, Julia; Vladis, Nathalie. (2017). Feel The Heat – A World Temperature Data Quilt, [image]. University of Edinburgh. EDINA. https://doi.org/10.7488/ds/1998.

Another theme was renewable energy – we included data from our School of Engineering on tidal turbines, and recent wave buoy experimental data:
The big 3-0-0-0: DataShare reaches three thousand datasets

All this raises the question – why bring these data together, what for? Do the datasets measuring and defining the problem really belong with the research working on technology to reduce greenhouse gas emissions? To answer this, I think the analogy of our other thematic collection on Covid-19 is apt. To develop and implement effective treatments and public health responses to Covid-19, we do need to understand a great deal about the cause, the pathogen and the pathology it creates. We should strive to break down barriers between domains of knowledge. So yes, to tackle climate change more effectively, we should all seek to better understand the underlying processes and the behavioural and technological solutions we must employ.

By bringing together research data from diverse teams in a single DataShare Collection, we empower the user to browse those datasets using the ‘facets’ feature in DataShare, or indeed a text search within the Collection. The user can filter by geography, by data creator’s name, by keyword, funder (see the screenshot below) or they can choose their own search term. When they reach an individual dataset, the breadcrumb trail at the top of the page can lead them into the original Collection where the dataset was first deposited, leading them to other work from the same research group, centre or School. This is one small way for the curation team to enhance the findability of the data. Scientists tell us there are challenges posed by the plethora of formats and programming languages used, even within disciplines. We hope that by making the connections and common themes between these different strands of research from different disciplines more visible, we make the data more findable, and perhaps hope to inspire new research questions or approaches.

a screenshot from the Collection page

DataShare’s facets

A word about DataShare’s structure: we find our depositors prefer to place their data in a Collection that reflects the organisational placement of their research group – typically the Collection represents the research group, and sits within a Community representing a research centre, sitting within a Community representing a School, sitting within a top-level Community representing a College:
DataShare structure

If you would like to suggest a theme for a new thematic Collection on DataShare, please contact the Research Data Support team:
Research Data Service | Contact

The RDS team, like all the University of Edinburgh’s teams, has a remit to address climate change as the university is committed to contributing to the UN’s Sustainable Development Goals, including no. 13 “Take urgent action to combat climate change and its impacts”:
Social Responsibility and Sustainability | The University of Edinburgh

We can all learn more about how we can take that urgent action effectively on the university’s amazing and inspiring “Climate Solutions” MOOC, available on edX:
Climate Solutions | edX 

I recommend anyone to take this course – it is free of charge, it’s fun, it is easy to fit around other commitments. I’ve nearly completed the coursework and already passed thanks to my quiz scores, got my nice PDF certificate signed by Professor Dave Reay. The Climate Solutions MOOC inspired me to create the Climate Change thematic Collection and it has really opened my eyes to the scale and nature of the challenge, and many actions we all need to take to contribute to halting the rise in global temperatures. Everyone has their part to play.

Pauline Ward
Research Data Support Assistant
Library & University Collections
University of Edinburgh

The big 3-0-0-0: DataShare reaches three thousand datasets

Quote

Confetti banner says "3,000th deposit!!!" 2021-08-02

Timestamp showing the accession of the deposit on the 2nd of August.

We’re thrilled Edinburgh DataShare has just ingested its 3,000th deposit:

Davey, Thomas; Draycott, Samuel; Pillai, Ajit; Gabl, Roman; Jordan, Laura-Beth. (2021). Wave buoy in current – experimental data, [dataset]. University of Edinburgh. School of Engineering. Institute for Energy Systems. FloWave Ocean Energy Research Facility. https://doi.org/10.7488/ds/3105.

The depositor was Dr Tom Davey, Senior Experimental Officer in the School of Engineering, who said:

“It is a pleasure for us all in FloWave to see one of our datasets achieve this milestone for Edinburgh DataShare. This is also the tenth DataShare upload making use of experimental outputs from the FloWave Ocean Energy Research Facility. Providing a reliable and accessible repository of our project outputs is not only important for our funders, but also promotes new research collaborations and builds lasting impact for our experimental programmes. This particular project will aid in the understanding measuring wave and currents at deployment sites for offshore renewable energy technologies, and adds to the existing FloWave portfolio of datasets in the field of wave energy, tidal energy, advanced measurement, and remote operated vehicles.”

You can explore more data generated at FloWave in the IES DataShare Collection:

Collection – The Institute for Energy Systems (IES) (ed.ac.uk)

Although the ‘wave buoy in current’ dataset is under temporary embargo, currently set to expire on the 5th of September, it is possible to request the data using DataShare’s request-a-copy feature in the meanwhile. Embargoes may be extended, or lifted early, usually reflecting publication dates.

You might also enjoy this hilarious and very popular video about FloWave:

 

Pauline Ward

Research Data Support Assistant

Library & University Collections

University of Edinburgh

How to DataShare

The latest in our series of Research Data Service ‘how-to’ videos provides guidance and handy hints on making data FAIR (Findable, Accessible, Interoperable and Reusable) by depositing in Edinburgh DataShare, the University’s open access data repository. This video is aimed at researchers seeking a Digital Object Identifier (or DOI, required by many publishers), wanting to share their data online and/or to archive their data somewhere safe for the long-term. The video demonstrates the DataShare submission interface, while the narration covers the kind of advice and guidance we would typically provide to users when working with them in person or via screen-sharing. Our hope is that users will find this video easier to access and quicker than having to request and schedule a meeting. That said, we’re still available and delighted to talk to users who have more complex questions or requirements about archiving and sharing their data.

Take a look at “How to Archive your Data on Edinburgh DataShare, the Open Research Data Repository” on MediaHopper

Pauline Ward
Research Data Support Assistant
Library and University Collections