On 2-4 November I attended the LibPMC Conference (International Conference on Performance Measurement in Libraries). The conference content was really varied, including a focus on the needs of stakeholders and communities and actively using qualitative and quantitative data to improve services and the user experience. Here are some highlights.

1. But can it last? How the pandemic transformed our relationship with data. Dr Frankie Wilson, Bodleian Library
2. Knowledge is PowerBI: How data visualisation helped inform services during the pandemic. Elaine Sykes, Liverpool John Moore’s University
3. Using return on investment to tell the story of library value and library values. Prof Scott HW Young, Prof Hannah McKelvey – Montana State University
4. Assembling a Virtual Student Library Advisory Board during COVID-19. Prof Chantelle Swaren, Prof Theresa Liedtka – University of Tennessee at Chattanooga
5. Charting the Change: Analysing How Online Delivery Made A Difference to Who Is Accessing Academic Skills Programmes. Louise Makin, Academic Engagement Manager, Liverpool John Moores University. 

But can it last? How the pandemic transformed our relationship with data.

Dr Frankie Wilson from the Bodleian Library, Oxford University holds the role of Head of Assessment, which replaced the historic role of ‘Bodley’s Secretary’ in collecting data for decision making across the Bodleian libraries. She reported that in the early years of her role, the data gathered expanded from that required by SCONUL to reflect every aspect of the Libraries’ activities; the annual statistics were communicated via colourful infographics, and senior managers were alerted to interesting trends. But “data remained a condiment, something served up on the side to add interest and even piquancy, not the main event.”

Then lockdown happened. A month later, “the demand from senior managers for data was insatiable”. In the volatile situation, with services being developed at unprecedented speed, managers needed real-time data to evaluate the impact new services and policies were having. There were two challenges in providing this data. Firstly, all the systems set up to gather data were based on an annual reporting cycle not daily updates. Secondly, the data was held in 22 different systems with no way of bringing it all together.

The solution, based on the assumption that it might be in place for a couple of months, was a manual process of extracting, cleaning and transforming the data and presenting it via a Tableau dashboard. The data was updated weekly as the process took 15 person-hours each week, rising to double that by November as more services were added. Qualitative data was collected as impact stories as well as quantitative data, gathered through the daily culling of stories on twitter, enquiries, live chat, and feedback to subject librarians. This work normalised data informed decision making and made it a key component running through absolutely everything. It demonstrated new evidence of library importance, e.g. it demonstrated reliance on Libraries by students and researchers by all disciplines.

Post-lockdown, the foundations remain that will support managers in using data to inform decision-making in the future. There is now the technical infrastructure to automatically extract data overnight from the 22 native systems, clean it, transform it, and present it. Managers have the skills to undertake analysis of the data and elicit insights. There is an awareness of the breadth and depth of data available – and a management expectation that it can be produced in a timely manner (previous work hid the full complexity of data acquisition, cleaning and presentation). Reporting frequency has reduced to quarterly and they are exploring automated data collection with better dashboard design.

Knowledge is PowerBI: How data visualisation helped inform services during the pandemic

Elaine Sykes, Liverpool John Moore’s University

They’ve learned that it’s important to closely define the research question at the start. Ours was ‘Who is using the library and what are typical usage patterns?’

To answer this they experimented with data visualisation software. While Tableaux is frequently used, there was an institutional decision to use PowerBI. Our data sources included bookings, headcount, gate count data. With this they generated patterns of usage, a barchart of no of bookings per time slot, visualisation of usage over time and occupancy measures – heat map. They were also able to understand who is using the library by Faculty, School, level of study and get average bookings per user by School.

They learned that:

• Students still preferred the library where their books were even when they didn’t have access to the books
• Analysing booking data vs gatecount data enabled library to identify issues. Solutions were not about library space but about adjusting booking parameters.
• Managers liked customisable reports

In order to do this they had to do a LOT of data cleaning which was labour intensive. Only quantitative data was used for this analysis. Qualitative comments in the NSS survey showed a gulf between student perception of library and service offers.

Future development to include:

• Creating an overarching library dashboard
• Analyse usage of new spaces
• Broaden scope of activities
• Establish better links with live data

Using return on investment to tell the story of library value and library values

Prof Scott HW Young, Prof Hannah McKelvey – Montana State University

As libraries face constant decreases to their materials budgets, it remains important that the role they play in removing barriers to information access is understood, and the value of access to library users is transparent. Using online journal and eBook subscription statistics, including usage and financial data, they conducted a return-on-investment (ROI) analysis to demonstrate the value of these subscriptions to our campus community through the lens of library professional values.

They first conducted an assessment involving ROI for library collections, comparing cost per use data for library subscriptions (Counter title usage reports, Yearly invoice data) with the cost to the student if the student had had to purchase that content themselves (Publisher pay per view cost (no subscription), –              Cost per unique item request (with subscription). The ROI study produced a dollars-saved result, which symbolizes the information access barrier that the library removes.

They were able to say that by providing access to information resources, the library had saved students $30 million over the last five years.

Assembling a Virtual Student Library Advisory Board during COVID-19

Prof Chantelle Swaren, Prof Theresa Liedtka – University of Tennessee at Chattanooga

During the lockdown period, a virtual student library advisory board was created to give students an active voice on library services and resources, support a diverse community and implement two-way communication. The board was launched in January 2021 and three meetings were held within the first semester.

• Larger than expected number of students participated. Students were attracted by direct email, ad on library website, social media.
• Student participants were given academic credit for participation. Also sent out library freebies to board members by post.
• Online meeting software’s (Zoom) tools worked well, including closed captions, text chat input, ease of transcription, and recordings, as compared with in-person meeting benefits.
• Soliciting participation from the campus’ Disability Resource Center community of students was successful. On application – asked students to self identify characteristics e.g. remote learners, first generation college students – to allow selection of representation
• The two main communication channels used: campus email and GroupMe, a mobile group messaging app owned by Microsoft. A student initiated the use of GroupMe, which resulted in nuanced differences between official and unofficial communications.
• A key theme was students sharing their love of the library
• 3 different ways to share feedback : In meeting – voice, In meeting – chat, Anonymous survey post meeting
• Significant drop in attendance over semester.
• Now having face to face meetings – supplying pizza
• Going to put library’s annual survey in front of students to see what they say

Charting the Change: Analysing How Online Delivery Made A Difference to Who Is Accessing Academic Skills Programmes

Louise Makin, Academic Engagement Manager, Liverpool John Moores University. 

As lockdowns were enforced across the UK due to the Covid-19 Pandemic, Library Services, Liverpool John Moores University (LJMU) quickly had to adapt its academic skills delivery programme from face to face teaching to online delivery. Managers were highly interested in understanding how this change in delivery impacted upon usage, both in terms of numbers and also demographics.

It was agreed that a dashboard be created in Microsoft Power BI data visualisation software. The design of the dashboards concentrated on two main groups of analyses.

The first compared the 2020-21 data with the 2019-20 patterns of usage. These analyses concentrated particularly on Semester 1 (Sep-Dec) as it was a good comparison of pre and post pandemic behaviours.

The second set of analyses focused on differences between the different interaction types; central teaching programme, embedded sessions, one-to-one sessions and feedforward and whether certain schools/demographic had different preferences for different sessions.

Comparing the usage data for both pre-pandemic and during, showed some interesting findings; demand for attending skills sessions increased by over 200% during the pandemic compared to the previous year. There were also some differences noted in the levels of students attending sessions, with the largest increase coming from postgraduate students.

From the 2021/22 academic year, usage will continue to be monitored and analysed closely, as the Skills offer begins to offer in person sessions again as well as continuing with an online offer. As a result, there is ongoing demand to have rapid information as to how the services are being used as LJMU adapts to the ‘new normal’. In addition, it is recognised that the current student cohorts have all experienced significant disruption to their educations, therefore it is hypothesised that there will be high demand for skills sessions. Managers want to monitor demand closely in order to provide the most tailored service offer possible. In particular, we look forward to analysing the demographics of attendees to the in person and the online skills offer.