Introduction
If a shared LMS were to emerge in Scotland, in whatever form it may take, would it enrich the user experience and benefit users as well as institutions?
To better understand the potential benefits for users of a shared LMS for Scotland, we wanted to get an idea of how users perceive the current LMS and use those perceptions and feedback to inform any future shared LMS. As there was an expectation that users would not be familiar with the Library Management System itself, the focused was on services, such as the Library catalogue, the discovery service and Aquabrowser which are supported by the LMS. We also wanted to gauge user awareness of existing services which allow users to search the holdings of other institutions and to find out if users are willing to travel outwith their institution to access resources.
Methodology
Staff and students are bombarded with various surveys throughout the academic year and, as a result, many may suffer from survey fatigue. Consequently, the intention was to make this a ‘quick and dirty’ survey in order to gather data quickly and as unobtrusively as possible- hence the vox pop style.
The survey was conducted over two days; first on 11th October and then on 28th November at two different library locations at the University of Edinburgh. A total of 62 valid surveys were completed.
The survey’s eight questions included a mixture of check boxes and open-ended questions.
In order to encourage participation, chocolate bars were offered in exchange for 10 minutes of time taken to complete the survey. Undoubtedly, in many cases, the chocolate was the overwhelming motivational factor and the timing coinciding, fortuitously, rather than by design, with participants’ mid-afternoon energy slump.
The project team would like to thank the large number of respondents who took the time to complete the survey.
Survey Results
Q1: Type of Library user
In total there were 62 survey respondents, the majority were undergraduate students; the rest were Postgraduate students. No visitors or academic staff completed the survey.
Figure 1
Q2: Length of time at the Institution
To gauge the level of respondent’s expected familiarity with the library and its services, respondents were also asked how long they had been at the University of Edinburgh.
Figure 2
Of the 28 users who had been at the University for less than one year, 19 were undergraduates and 9 were postgraduate students.
Q3: How familiar with and how often do students use existing LMS supported services?
Library users were asked if they had used any of the following to find things on their reading list or for their study and research:
- The Library catalogue
- Voyager
- The OPAC
- The Library Website
- Aquabrowser
- Searcher (Searcher is the service name for the University of Edinburgh’s EBSCO Discovery service)
All these services provide access to the library catalogue. This question was intended to find out which methods of accessing the catalogue are most familiar to users and which are used most often.
Users were asked to select one of the following four options for each resource:
- No, never heard of it
- No, heard of it, but never used it
- Yes, used it once or twice
- Yes, I often use it
Undoubtedly, Q3 was a bit of a trick question and also a tricky question. It was obvious on several occasions, that when faced with this list, users were unsure of the terminology or ‘label’ applied (by the library) to the services they use.
Figure 3
Almost half of the respondents, both postgraduate and undergraduate, have been at the University for less than a year which may explain their unfamiliarity with some of the LMS-related resources on offer.
There was a firm expectation that users would not recognise, ‘Voyager’ as the LMS currently in use at the University. To my knowledge, it is not promoted in any way and the assumption was that the LMS itself is irrelevant to users. This proved to be the case as 82.3% of respondents had never heard of Voyager.
Overall the Library catalogue emerged as the most used LMS supported service on the list: 38 respondents said they often used it to find items on their reading list or for their study and research. Reassuringly, only one individual had never heard of the library catalogue.
The OPAC was included as an option to test awareness of terminology. The expectation was that this would be slightly more recognisable than Voyager to users. Although the majority of people (46) had never heard of the OPAC, 12 people had heard of it and four people said they used the OPAC, of these, three said that they used it often.
Having previously conducted usability surveys, I was also aware that users can confuse the Library website with the Library catalogue. This survey doesn’t indicate if users made a distinction between the two. However, it is interesting to note that the Library catalogue is still used, at least by this group of students, more often than the Library website.
The Library website provides access to the Library catalogue and is a gateway to e-resources. As the library webpage is consistently one of the most viewed University webpages, I expected participants to be familiar with it. In fact, it is the second most popular choice, after the Library catalogue, amongst participants for finding items on reading lists and for study and research. 53.2% of participants say they often use it to find items on their reading list or for study and research.
The University of Edinburgh also provides access to the Library catalogue via Aquabrowser which provides a more Google-like discovery layer to search the catalogue. Surprisingly, given its arguably more user friendly interface, of the 40.3% who are aware of Aquabrowser, only 11.3% of respondents said they often use it. In fact, over half the respondents, 56.5% , had never heard of Aquabrowser. However, it is impossible to tell within the scope of this survey if this is due to lack of promotion or usability.
Searcher is the University of Edinburgh’s branded EBSCO Discovery Service. Users can search the Library catalogue and access e- resources via Searcher. Searcher emerges as the third most used resource amongst participants. More students have heard of the Discovery Service than Aquabrowser.
Q3 Summary
The Library catalogue emerges as the most used resource. Is this because for new and old users alike it is a familiar and recognisable resource? Is it because during the first months of study this is the LMS-related resource promoted most heavily to new students or, is it simply because it is the most effective tool for finding books and journals required for study and research? However, does emerge is that the Library catalogue, from the list of the various tools on offer, is the most recognised and most used LMS-related service.
Q4: From the list in Q3, which resource do you find most useful?
The fourth question had four parts and was designed to gather feedback on the resource the respondents found most useful and to find out which system users prefer to use to find library resources.
- Question 4b asked respondents what they liked most about the resource
- Question 4c asked respondents what they liked least
- Question 4d asked respondents what would make this resource better
Question 4a asked respondents to select the resource they found most useful from the list provided in Q3.
Figure 4
53.2% of students surveyed chose the Library catalogue as the most useful resource. 17 were students who have been studying at the university for less than a year.
Searcher was selected by 22.6% as the most useful resource and the Library website was chosen by 14.5% as the most useful resource. The person who chose the OPAC as their favourite resource, answered, ‘library catalogue/OPAC’ suggesting he/she is aware they are the same thing. Unsurprisingly, no-one selected Voyager as the resource they found most useful.
LIBRARY CATALOGUE
What do users like best about the Library catalogue?
33 users selected the Library catalogue as the most useful resource – 22 comments referred to the ease of use and simplicity of this resource.
‘…it gives me the results I need..’
‘…[it] helps me find what I’m looking for nearly all the time.’
‘All I need is there’
The remaining comments mentioned that the Library catalogue is ‘easy to find’, ‘reliable’ ‘efficient’ and ‘quick’.
What do users like least about the Library catalogue?
The majority of the comments recorded relate to the catalogue search not being intuitive.
“The fact that you can put in accurate information and still yield no results”
“Sometimes don’t find books although I have entered the ‘right words’ “
“The way your search may come with no results if you use words that upsets the search eg. ‘the’ at the start.”
“It often produces irrelevant results, too many results, not the results I want”
Students also thought that the look and feel of the interface could be improved.
What would make the Library Catalogue better?
The replies logically followed on from what they liked least about the Library catalogue and suggest improvements to the interface and search. Suggested improvements include:
- More search fields
- Genre specific searches
- More intuitive search tool
- Easier Log in
- Clearer location and availability information
- Better advanced search options
However, four students comment that there’s nothing they’d improve about the catalogue as it works well as it is.
“I don’t think improvement is needed”
“I think it’s already pretty useful”
Library Website
What do users like best about the Library website?
Nine students surveyed said that the Library website was the resource (from the list provided) they found the most useful.
For 4 of the 9 students ease of use is what they like most about this resource. For two they like it best because it provides easy access to other electronic resources. Two mention it has a ‘good user interface’ and is ‘easy to navigate’ for two it’s the information on how to use the library, in particular opening time, which is most useful.
“Quick information, very easy and well explained”
“It gives easy access to journals and e-resources”
“Information about how to use the library, opening times etc”
What do users like least about the Library website?
Five comments were recorded. Three mention that the website can be slow to use and the remaining two said that resource can be difficult to find.
What would make the Library website better?
Six comments were recorded covering design, search functionality, ease of access and currency. Suggested improvements include:
- Better search facility/search bar
- Easier access, such as automatic log-in
- Could be updated more often, maybe more info on how best to use the library, study spaces available
- Better organization
AquaBrowser
What do users like best about Aquabrowser?
The three students who said Aquabrowser was the resource they found most useful left comments. Ease of use and returning relevant resources, as well as allowing for the discovery of new resources, are mentioned.
“It brings up every pertinent source the library has”
“Quick easy to use and gives extra suggestions at the side”
“It’s so easy to use and often suggests resource[s] I’d never heard of “
What do users like least about Aquabrowser?
Only two comments were recorded here. One to say, ‘nothing really’ and the other commenting, ‘Sometimes I’ve had problems with regards to getting too many unrelated results which makes it harder to get what I want’.
As with the library catalogue the ability to intuitively refine searches and manage results is important to users.
What would make Aquabrowser better?
Only two suggestions were made: the first, ‘more electronic resources’ and the second student said,” I generally find it [Aquabrowser] very helpful, but maybe some kind of equivalent technology for journals and articles would be amazing.”
Searcher
Searcher (the University of Edinburgh’s branded Discovery Service) is the second most popular resource from the list provided. Searcher was chosen by 14 students as the most useful resource. The discovery service with all its additional functionality is still only half as popular as the catalogue- at least amongst students surveyed here. However, the reasons why students like it best are the same as the catalogue; ease of use, it finds what you need and it’s quick.
What do users like best about Searcher?
9 of the 14 comments mention Searcher is easy to use. In addition, four students say they like the way it allows them to search and access a variety of resources. Two mention how convenient and easy it is to access via the student portal and two that it’s a good research tool.
“Easy to use, simple set out. Finds what you are looking for easily, no trawling through pages upon pages”
“Gives journals along with books. It always tends to find useful/relevant journals in the search topic. Easy to use.”
“I like how it searches a variety of resources. It’s not the most in-depth but it provides a great foundation for research.”
What do users like least about Searcher?
Eleven comments were recorded here, six of which refer to locating and accessing items. Three users mention that they don’t like to search and find results for content to which they can’t access (because the University doesn’t hold a subscription). As with the catalogue, location information could be clearer.
“It shows you all the articles that Edinburgh doesn’t subscribe to, rub it in”
“Sometimes shows articles that would be useful but aren’t available”
“Often I am confused as to where to actually find the book I am looking for”
What would make Searcher better?
More of everything! Subscribing to more resources, a greater variety of resources, more full text and more search fields would all improve the discovery service.
Q5 Please list the resources (systems, databases or websites) you use most often to find items on your reading list or for your study and research
Q5 asked students to list any other of the resources they use most often to find items on their reading lists or for study and research. This question was intended to find out what other resources students use and if what, if any, resources emerge most popular.
The majority of respondents left an answer for this (57 of 62). There is no surprise that Google is the top ‘other’ resource used by students. However, when asked what other resources they use, the catalogue is the resource listed most often after Google, and other services supported by the LMS, Searcher and Aquabrowser, appear in the top ten.
33 students stated the library catalogue was the most useful resource and a further 14 listed it as a resource they often use. This suggests that for 75.8% of students surveyed, the library catalogue is at the core of their research and study. No other resource is cited by as many users.
Q5. Resources used most often to find items on reading lists or for study and research: |
No of including resource on list of ‘most used’ |
|
|
15 |
|
14 |
|
11 |
|
8 |
|
7 |
|
6 |
|
5 |
|
5 |
|
4 |
|
4 |
|
3 |
|
2 |
|
2 |
|
2 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
|
1 |
Figure 5
Q6. Have you ever used any of the following following services: WorldCAT, SUNCAT, SALSER or COPAC?
This question was to gage awareness and find out how many of the students make use of existing services which offer access to catalogues and collections outwith their own institution.
Figure 6
SUNCAT’s Benefit and Impact Survey 2012 shows that 81% of users identified themselves as Librarians or Information professionals. 9% were postgraduates or researchers and only 8% were undergraduates.
http://www.suncat.ac.uk/news/docs/SUNCAT_Survey_Report_Website_March2012.pdf
Given the large number of undergraduate respondents to this survey it is perhaps unsurprising that few were aware of or used these services.
However, the respondents who said they often used the services, or had used them once or twice, were all postgraduate students.
Nevertheless, this result may also suggest that services which facilitate discovery of other collections need to be more effectively promoted- at least at this particular University.
Q7. While you’ve been a student, researcher or member of staff here, have you ever visited another library, outwith the University of Edinburgh, for your study or research?
This question was to get an idea of how many students are currently making use of libraries other than those belonging to their institution.
Figure 7
Considering the majority of respondents were undergraduates it is surprising that a third of all respondents have visited other libraries. 27.5% of undergraduates and 63.6% of postgraduates who took part in the survey had visited another library. The survey didn’t ask why students had visited another library or which libraries were visited.
Q8 Asked if users would be prepared to travel to another library outwith the University of Edinburgh for your study or research?
Bearing in mind that while services, such as those mentioned in Q6, allow users to search and discover content in other collections, they are still required to physically go to the library to access items or to request items via Inter Library Loans.
This question was intended to gauge users’ willingness to go beyond their institution to access collections.
Figure 8
71% of students surveyed would be prepared to travel to another library for their study or research.
Of those 17 who would not be prepared to travel; 15 were undergraduates and 2 postgraduates.
Three unsolicited comments were left by students showing varying levels of enthusiasm for travelling to other libraries:
“Of course!”
“[yes] but prefer one stop shop”
“[yes], but rarely, if I had really had to..”
Summary
As expected, users are not familiar with the name of the LMS the library uses. The term OPAC is also unfamiliar. Users are more aware of the services with which the LMS integrates and supports.
Of these, for the students surveyed, the Library catalogue is the most familiar resource. When asked, the Library catalogue is the most useful resource for users. Results suggest it remains at the core of their learning and research.
Whatever the user’s preferred resource, the reasons why are consistent. The most useful resources are: easy to use, quick and find what users need.
The improvements which could be made are also consistent across services: a better designed interface, more intuitive searching, clearer location information and more options to search and refine results.
There is very little awareness of existing services which allow other catalogues and collections to be searched. However, awareness and use of these services is greater amongst postgraduates.
Comments from the survey suggest that, when using the discovery service, users don’t like to find results for content to which they don’t have access. Yet, almost two-thirds of students said they would be willing to travel outwith their institution to another library. This was unexpected and raises some interesting questions about user expectations. Do uses expect to be able to access everything they need in the one library or institution? Is the need to visit other libraries an expected necessity and are users just resigned to the inevitability that their own institution won’t always have everything they need? Or, do user expectations towards accessing physical items differ from accessing electronic content?
Based on users’ perceptions of existing LMS-related services, new shared LMS services or discovery layers, should be:
- Easy to find
- Easy to log in to
- Easy to use
- Quick to use
- Have a well-designed interface
- Intuitive to search
- Return relevant results
- Allow search results to be refined
- Search across a variety of resources
- Help users discover related materials
- Return results to which users have access
- Provide clear location and availability information