Performance Measurement & Metrics

Experience assessment: designing an innovative curriculum for assessment and UX professionals

Performance Measurement and Metrics, Volume 19, Issue 1, Page 30-39, February 2018.
Purpose While assessment and user experience (UX) have been identified as areas of growing focus in all types of libraries, there is currently little infrastructure to prepare students for these roles (Applegate, 2016; Askew and Theodore-Shusta, 2013; Nitecki et al., 2015; Oakleaf, 2013; Passonneau and Erickson, 2014). As a step toward addressing this gap, a team from an American Library Association-accredited master’s program situated at a large public land-grant institution (LGU) worked with practitioner partners from academic libraries and information agencies to develop a new model for preparing information professionals with assessment and UX expertise. The paper aims to discuss these issues. Design/methodology/approach In fall of 2015, faculty members applied for funding from the US Institute for Museum and Library Services Laura Bush 21st Century Librarians program for a program to develop formalized assessment and UX training in Library and Information Science (LIS) education. The student cohort would have interests in two areas: academic libraries and specialized information agencies. The two groups would complete much of the same coursework, earn the ALA-accredited master’s degree and have the opportunity to engage in co-curricular activities focused on UX and assessment. However, each sub-group would also pursue a subject-specific curriculum. In April 2016, IMLS funded the program. Findings In addition to reviewing the literature related to best practices in curriculum development, the authors describe the process of designing the program, including the curriculum, co-curricular mentoring and practicum opportunities, and the tools developed to evaluate the program’s effectiveness. Research limitations/implications At a time in which the library practitioner and LIS educator communities are contemplating how best to prepare professionals with much-needed expertise in assessment and UX, UX-A represents an innovative approach in professional preparation. Although the UX-A program is grant-funded, several of the program components could be adapted and incorporated without such support. Originality/value This paper discusses the structure and history of the program, issues related to developing a new curricular program for LIS education, and the educational and professional development needs of the assessment and UX professional community. It includes an extensive review of literature related to LIS curriculum development, practica, and professional mentoring, as well as suggestions for implementing elements of the program in other settings.

Performance Measurement & Metrics

The use of academic libraries in turbulent times

Performance Measurement and Metrics, Volume 19, Issue 1, Page 40-52, February 2018.
Purpose The purpose of this paper is to explore how an innovation in the University Management Information System was leveraged to incorporate library data by an initially sceptical strategic management team. The rationale was to extract evidence of correlations between library use and student achievement. This kind of information is of particular interest to the institution, which is at present dealing with crises popularly summarised in the slogan “#FeesMustFall” among students who suffer from the effects of poverty and exclusion in higher education. Comment is offered on some of the relationships between student library behaviour before, during and after the nationwide disruptions that destabilised universities and threatened their survival at the end of 2016, just before the final examination period. Design/methodology/approach Data were extracted from the data warehouse from the comparative demographic perspectives of students’ degrees of disadvantage in an effort to uncover any hitherto hidden patterns of library use. Findings The use of the library as expressed by footfall and loans was mapped against students’ pass rates and their collective GPA, indicating that increased library use correlates positively with better academic performance. Some of the initial correlations between student library behaviour before, during and after the nationwide disruptions that destabilised universities and threatened their survival at the end of 2016 just before the final examination period are explored. The effects that library closures (under threat of damage) at a critical time in the academic year might have had on library use and on student performance are interrogated. Practical implications Students on financial aid, which was used as an indicator of disadvantage, come from schools and environments where access to information technology and libraries is very limited, so that library habits are either poorly established or not at all. At the University of Cape Town (UCT), considerable support is in place for students to encourage the development of library habits. An analysis of available data indicates that students who have acquired library habits regardless of unfavourable financial circumstances do not exhibit behaviour and academic outcomes markedly different from that of their more privileged peers. Originality/value Combining library data with data from the university data warehouse is a new approach in South Africa. It is an approach that is of value both to the library and the institution at large and has brought meaningful insights into the role the academic library might be seen to play in promoting student academic achievement.

Performance Measurement & Metrics

Articulating value and impact through outcome-centered service delivery

Performance Measurement and Metrics, Volume 19, Issue 1, Page 2-11, February 2018.
Purpose The purpose of this paper is to share the ways in which student and learning support at the University of Sunderland has embedded and matured a new outcome-centered performance model – the Quality Model – in order to create an agile evidence-base of value, outcome and impact evidence. The authors will also share how, having established the fundamental principles regarding value and impact capture in our library setting, the concepts and approaches have also been developed and applied successfully within the context of multi-converged service delivery across the wider student and learning support service, using the AMOSSHE Value and Impact Toolkit. Design/methodology/approach The authors’ approach will be illustrated with two case studies, the first focusing on the university library services and the second on the student counseling service. Findings The findings will reveal that by establishing an outcome-focused model, the authors have been able to apply it across a converged service in order to generate the evidence required to articulate the value and impact of our key service objectives. Originality/value As a performance approach, the Quality Model is an original concept in that it is a self-formed model designed to meet the strategic needs of the University of Sunderland. It differs from many performance models in that it is founded on a holistic approach to service culture and customer-relationship management and is based upon strategic marketing principles. The AMOSSHE Toolkit is a pre-existing toolkit which is fully referenced in the paper. The Quality Model and AMOSSHE Toolkit are of particular relevance as many higher education services are increasingly challenged to demonstrate their value and impact and the outcomes their services deliver. This calls for a strategic approach to managing qualitative evidence. Therefore, although bespoke, the approach is transferable to the strategic priorities of other HE settings.

Performance Measurement & Metrics

Academic libraries and student learning outcomes

Performance Measurement and Metrics, Volume 19, Issue 1, Page 75-87, February 2018.
Purpose The purpose of this paper is to examine the literature on student learning outcomes and academic libraries. An analysis of 81 papers revealed the following themes: outcomes assessment of library skills instruction, tools for assessing students’ achievement of learning outcomes, institutional accreditation and student learning outcomes, academic libraries’ impact on student learning outcomes, and the creation of learning outcomes. An understanding of these themes highlights how student learning outcomes are created, assessed, and valued in the academy. Design/methodology/approach The methodology centered on a literature review of student learning outcomes and academic libraries from 2001 to the present. To identify relevant articles on this topic, the authors searched the library literature with various combinations of the following keywords: learning outcomes, library, academic, university, college, and higher education. Findings An analysis of 81 papers on student learning outcomes and academic libraries revealed the following five themes: outcomes assessment of library skills instruction, assessment tools for student learning outcomes, institutional accreditation and student learning outcomes, academic libraries’ impact on student learning outcomes, and the creation of learning outcomes. Research limitations/implications The research was limited to articles published in English and after 2000. Practical implications The findings can be utilized to inform instructional librarians about creating and assessing student learning outcomes. In addition, the paper can assist librarians in forging partnerships with academic departments and faculty in creating student learning outcomes to support course and departmental outcomes and accreditation efforts. Originality/value The research offers librarians opportunities to incorporate student learning outcomes in library instruction as well as collaborate with faculty in creating student learning outcomes.

Performance Measurement & Metrics

Contextualizing library assessment within a broader ecosystem

Performance Measurement and Metrics, Volume 19, Issue 1, Page 18-29, February 2018.
Purpose The purpose of this paper is to draw a picture of the current landscape of library assessment based on the data gathered in interviews. The authors will focus specifically on the continuum between micro and strategic assessment and share the lessons learned from diverse institutions and geographies about how to build a culture of assessment. Design/methodology/approach Between 2015 and 2017, the researchers have interviewed more than 75 library directors and leaders, library assessment practitioners, and academic experts on four continents about library assessment and its current state in their institutions. Findings The results reveal a varied landscape, with libraries in widely varying stages of assessment performance and readiness. Originality/value This paper presents the results of a large-scale study over more than two years. More than 75 people have been interviewed in five countries. The scale and scope of the work is both significant and unique.

Performance Measurement & Metrics

Generating bespoke value and impact evidence to inform a thought leadership approach to service engagement at The University of Sunderland

Performance Measurement and Metrics, Volume 19, Issue 1, Page 53-65, February 2018.
Purpose The purpose of this paper is to share how the maturing value and impact model at The University of Sunderland is enabling the author to generate evidence and articulate the insights. The author draws from that evidence to inform and underpin the strategic service planning, resourcing and reporting. Design/methodology/approach The author will demonstrate how impactful articulation of these insights through data visualisation is enabling the author to employ thought leadership in the relationship management with stakeholders by increasing their understanding of the benefit of engagement with the service offers and demonstrating value for money and the value and impact of the role in achieving institutional objectives. The paper will give an overview of the key techniques of the model and will then demonstrate its practical applications using the following examples: how the model is underpinning the approaches to thought leadership in relationship management by enabling the author to effectively generate and articulate evidence to inform strategic faculty action plans; how the model has enabled the authors to develop a new graphical approach to annual reporting. By combining the variety of data sets generated by the model, the author is able to articulate the outputs and impacts of cross-service holistic service offers and clearly demonstrate how institutional strategic objectives are fulfilled. Findings The author will discuss the key findings including: the importance of embedding the model at the heart of the service culture – both in terms of involving staff in data generation and of developing an evidence-based culture of service planning; the benefit of meaningful data, analysis and insights in helping to inform and underpin strategic conversations and relationship management; the transferability of the model across service settings; the agility of a snapshot approach in enabling the authors to evidence and inform current strategic service priorities; the impact of a “rounded narrative” technique in articulating powerful human insights which demonstrate engagement, impact and value; and the importance of creative data-visualisation techniques in communicating the insights for maximum impact with the customers and stakeholders. Research limitations/implications This case study demonstrates the approach taken to fulfil a specific strategic need at one UK HE institution. Therefore, the readers are encouraged to consider the approach within that context. Originality/value This paper shares how a strategic approach to capturing and communicating value and impact evidence can contribute to thought leadership in articulating library impact.

Performance Measurement & Metrics

The Curtin Materials Availability Survey 2017

Performance Measurement and Metrics, Volume 19, Issue 1, Page 12-17, February 2018.
Purpose The purpose of this paper is to extend the principles of earlier print-based availability surveys to the context of today’s electronic library, and explores the question of an appropriate methodology. The ability of clients to find what they want remains a central question, as does the library’s ability to identify and address the reasons that clients fail to find what they are looking for. Design/methodology/approach Catalogue users at Curtin University Library were invited to complete an online survey indicating whether they had found the electronic item they were looking for, and if not to nominate the reason why. Responses were then verified and analysed by library staff. Findings The survey attracted a low number of usable responses, though the proportion of respondents who stated they were able to find what they were looking for was consistent with the findings of earlier studies. It was possible to identify a small number of cases where the library did not hold the item required, though most failures were either due to technical reasons or could not be fully investigated because not enough information was provided by the respondent. Research limitations/implications The survey conducted was inconclusive, partly because the delivery method used was quite cumbersome, and also because it focussed on known item searches rather than topic searches. The paper includes suggestions on how the survey could be broadened and technically improved. Originality/value The paper shows the value and limitations of conducting a materials availability survey in the electronic library, and makes suggestions on how the effectiveness of such a survey can be maximised.

Performance Measurement & Metrics

COUNTER: Release 5 of the Code of Practice

Performance Measurement and Metrics, Volume 18, Issue 3, Page 180-186, November 2017.
Purpose Librarians need to measure returns on the investments that they make, and for digital content, the COUNTER Code of Practice for usage statistics is an important metric. The paper aims to discuss these issues. Design/methodology/approach This paper is a viewpoint. Findings Release 5 of the Code of Practice has been developed to make it easier for publishers to be compliant. It can be adapted and extended as digital publishing changes over the years. Originality/value Release 5 consists of four master reports, a reduced number of metrics and additional attributes. These will enable librarians to roll up or drill down through reports with ease, eliminating the need for the kinds of special-purpose reports required previously.

Performance Measurement & Metrics

The psychometric properties of an information-ethics questionnaire

Performance Measurement and Metrics, Volume 18, Issue 3, Page 166-179, November 2017.
Purpose The purpose of this paper is to provide an overview of the development and validation of a self-report structured questionnaire based on the theory of planned behaviour (TPB). The questionnaire was used to investigate university undergraduate students’ cognition, behavioural intent, and behaviour concerning the ethical use of information and communication technologies (ICTs). Design/methodology/approach A quantitative correlational cross-sectional approach to data collection was used by administering a self-report questionnaire to a sample of 327 undergraduate students. To establish the construct validity of the questionnaire, internal consistency reliability and factorial analyses were performed. Findings Significant but different correlations between the constructs were under study. The behavioural intention had the greatest and most significant correlation with behaviour pertaining to information ethics. Overall, the scales, which constitute the instrument, showed acceptable indices of measurement validity. Practical implications The research implications of this paper shed light on the applicability of the TPB to the specific context in which this study was conducted. The results imply that the TPB framework can be extended and employed to understand better the influence of comprehending information-ethics concepts on the intentions and practices related to the ethical use of ICTs. Originality/value The paper has methodological value for researchers who seek to develop empirical instruments for measuring the factors that bear upon the attitudes and the behaviours related to information ethics. Furthermore, the paper has pedagogical value for students, teachers, and developers of information-ethics educational programmes at the tertiary level.