Tag Archives: Metadata Games

CSV Mind-Blow(ve)n!

Uncategorized, , , , , , , , ,
PSV Eindhoven - Eredivisie champions
Never waste a football-related pun. It’s been a good week for both PSV and CSV.

I was lucky enough to have a paper accepted to Csv,conf,2* in Berlin on the 3rd-4th of May, which was great to do, but it also got me through the door to see loads of great things going on in data and its surrounding technology. Yes, there was heavy mention made of CSV and Spreadsheets; in fact, at times it was akin to an AA meeting ,with people guiltily admitting their love of Excel. This left me feeling- quite worryingly- vindicated in a lot of the things I do.

As is always the case with any conference review blogpost, it’s not viable to list every link or ruminate on the message of every talk, so I’ll just home in on a few highlights. The talks (available in slide or video form) are appearing over at Lanyrd.com, and they’ll give a lot more depth to what was spoken about.

My own reason for being there- as far as my talk was concerned- was to look at better ways of processing workflow, enriching data, and improving engagement with our collections. Afterwards, I had some interesting conversations: I was alerted to the tool NeuralTalk2  by Maciej Gryka of rainforestqa.com, a company that specialises in cleaning data using mechanical turks and crowdsourced test-cases. Neural Talk, though, is a captioning tool, which will attempt to visually recognise what your image is “of”. I’m sure it fails as much as it succeeds, but, as I pointed out, we’ve not really used this kind of tech to enhance our metadata, so there’d be no harm in running some of our images through and seeing what it comes up with.  Another chat, with a lady from UC Santa Cruz, made it clear that we are quite liberal with our approach to crowdsourced data. Where we have generally decided it’s fine to surface as long as it is properly marked as such, they are proceeding rather slowly, due to a particularly strict metadata librarian.

The keynotes were deliberately intended to cover a range of disciplines that might be new to most people at this highly eclectic conference. Resultantly, there were interesting talks on technology and activism (including visualisations of the Ebola crisis and police brutality); ethics in technology and workflows to give your consent without clicking on unreadable terms and conditions (do you know what SmartBins are taking from you as you pass?); dealing with messy spreadsheets (the Enron crisis showed this institution to manage them terribly), and open data with neuroscience (a lot of mouse brains in action).

Some other tools that we could be looking at:

Zegami – great for exploring large banks of images and spotting patterns across them. Can it work with IIIF, I wonder?

OpenRefine– a tool that we perhaps should have been using for some time to rationalise spreadsheet data, which could certainly save lots of time. Our former colleague, Richard Jones of cottagelabs, is a great advocate of these kinds of tools, as his talk made clear.

DataBaker– created in collaboration between the Office of National Statistics and ScraperWiki. This Python application can convert any ‘pretty’ spreadsheet into usable source data in CSV.

CSV Rinse and Repeat– built in Paris by Mathieu Jacomy of the Paris MediaLab, this is a JavaScript tool which intends to cut down the distance between data, coding and visualisation. Basically you take your data, spot patterns, recode to surface interesting things, and generate a visualisation out the end, in an iterative process.

Wikipedia Googlesheets– I am not sure if we would have a use for this specifically, but it was fascinating to see a plugin which serves up spreadsheet formulae coded in Google Apps JavaScript, which can then be used to interrogate any Wikipedia pages. Particularly of note is the ability to combine category pages and pageviews, to see if real-time events are influenced by Wikipedia and vice-versa. Developed by Thomas Steiner at Google.

Finally, here are three interesting observations, which certainly struck a chord with me:

  • It is now deemed quite acceptable, as a symptom of rapid development, perhaps, that the CSV is used as the master dataset; perhaps the file-based database’s day is not over. I certainly found out about some interesting applications built in this way.
  • I heard no-one but myself talk about Excel macros- at a spreadsheet conference, no less! It is far more fashionable these days to read your data as csv, and code against it using JavaScript, R, or Python. I clearly need to get out of the 1990s.
  • EVERYONE suffers from problems with diacritics, glyphs, badly formatted data and what happens when you import a CSV into a spreadsheet tool that tries to be too clever. It is not just me.

All in all, an excellent couple of days, which have filled me with ideas for improvements for existing workflows. Hopefully the likes of the DIU will reap some benefits!

Berlin Dom und Fernsehturm
Insert standard caption regarding “old content meets new technology to surface it”!

Scott Renton, Digital Developer

*The commas are intentional, by the way!

Where’s Dolly? Another metadata game

Uncategorized, , , , , , ,

DollyTile

The Towards Dolly project, showcasing research from genetics at the Roslin Institute, has helped surface lots of interesting material within the library, not least at the current exhibition. One of its outputs was the Science On A Plate subproject, a Wellcome-funded digitisation of 3,500 glass plate slides.

We’ve put these into our metadata games infrastructure, to allow social tagging of the slides, and- while they are well-described- improve their keywording to allow superior faceting in the LUNA application, making the collection more searchable. The game follows the same methodology as the Tag It! Find It! art game- tag ten images, and vote on ten, and build up a high score. It has the additional incentive though, of a hidden Dolly The Sheep image. The lucky user who comes across this image first will be lavishly rewarded (i.e. they will almost certainly get an L&UC calendar!)!!

The game was launched in time for a talk given by Gavin Willshaw, Clare Button and Claire Knowles at The Central Library– a great example of our Library engaging with the wider community- on Tuesday September 29, and will be featured again at Ada Lovelace Day on October 13. You don’t need to be at these events to play the game though- it’s there for you to have a go at any time you like, at librarylabs.ed.ac.uk. You’ll need to login via EASE.

Scott Renton- Digital Development

Screen Shot 2015-10-01 at 12.27.22 copy

Open Repositories 2015

Uncategorized, , , , ,

Last week I was very fortunate to attend the Open Repositories Conference in Indianapolis. Where I was co-authoring a presentation on metadata games with Tiltfactor, Dartmouth College, and also co-chairing the new Developer Track and Ideas Challenge at the conference with Adam Field from EPrints Solutions, University of Southampton.

IMG_2661The conference hotel in downtown Indianapolis

It was very packed few days as the conference has moved from five to four days with more tracks running simultaneously. This unfortunately meant I didn’t get to see all the presentations I want to see, but all the papers are online and the main conference room was recorded. The keynote speakers were Kaitlin Thaney (Mozilla Science Lab) and Anurag Acharya (Google Scholar) who both gave very interesting and very different presentations. Kaitlin managed to hold the whole room’s attention despite having to talk via Skype, due to problems with flights. Their slides are available here and the videos should be online soon. Anurag’s talk was very specific on how repositories need to be designed to work with Google Scholar and has already created lots of debate over the use of PDF cover pages.

The Developer Track was new for OR15, it was designed, along with the Ideas Challenge, to stop the developers who attend the conference being torn between writing code for a competition and attending conference sessions. There were lots of demos and it was great to see no one having to apologise for screens of code, XML and terminal windows. I found the sessions really practical and have a list of things to try now I am back in the office, starting with Hardy Pottinger’s Vagrant-DSpace.

IMG_2700Hardy demoing Vagrant-DSpace

The Ideas Challenge, was designed to be less time intensive that the previously run Developer Challenge but still encourage people to discuss issues they would like to resolve, meet new people, and have a fun session where audience participation was encouraged. Adam and I created an example challenge and solution based on the Sound of Music Idea’s Challenge Slides. We had 9 entries to the challenge this year and thanks to Adam’s scoresheet there were no long deliberations and a clear winner was identified after our judges handed in their scores. Congratulations to the winners Blog post about the challenge winners.

IMG_2780Slide from the Winning Entry – Email Deposit

My presentation on Crowdsourcing Metadata Session Slides was in front of a packed room. Helped by my talk being sandwiched between Rob Sanderson from Stanford University and Simeon Warner from Cornell University. I got some great questions and hopefully passed on the message that engagement is just as important as developing the tools. My co-authors at Tiltfactor launched their new OCR verification games last week http://beanstalkgame.org is very addictive (you have been warned).

Screen Shot 2015-06-19 at 06.14.56Tiltfactor’s Beanstalk game

There was a lot of discussion during the Monday’s DSpace DCAT and Committer meeting and also during the DSpace Interest Group about the ambitious work planned for DSpace 7 in 2016. The University of Edinburgh manages DSpace repositories on behalf of the Scottish Digital Library Consortium and Stuart Lewis is a DSpace Steering Committee member. I hope that these plans get the support they require within the DSpace community to make them happen.

Hardy Pottinger has written a blogpost about the conference and his ’stuff to check out later’ to take away from the conference and try is similar to mine so I won’t repeat it. Although I would add Vagrant and implementations of IIIF. Both of which have been on my todo list for a while, but I really really want to find time to play with them now.

Thanks to Sukie at Tiltfactor and Adam for lots of Skype calls over the past few months, and to the Open Repositories organisers for a great week.