Noticias em eLiteracias

✇ Performance Measurement & Metrics

Global Libraries impact planning and assessment progress: part 2

14 de Junho de 2019, 02:42
Performance Measurement and Metrics, Ahead of Print.
Purpose The purpose of this paper is to report on recent performance measurement and impact evaluation progress made in Brazil, India, Indonesia, Moldova, Turkey and Vietnam as part of the last phase of the Global Libraries Initiative. Design/methodology/approach The country reports are presented as a series of case studies, in some cases supplementing those reported earlier in this journal. Findings Recent country-specific survey findings are reported and some conclusions are offered. Research limitations/implications This paper demonstrates how the adoption of a common approach to library service evaluation across several countries can strengthen research practice at country level beyond the Global Libraries Initiative itself. Practical implications This paper shares Global Libraries IPA learning at country level with people in other countries who may be contemplating public library evaluation at regional, national or local levels or who are interested in performance measurement and impact evaluation. Social implications The paper shows how focusing on the impact of public library services on users can enhance the understanding of community requirements and inform the development of more effective services to library users and communities. Originality/value These case studies reflect concentrated impact evaluation and performance measurement work at country level across a range of countries over more than 18 years.
✇ Performance Measurement & Metrics

Impact evaluation and IFLA

14 de Junho de 2019, 02:40
Performance Measurement and Metrics, Ahead of Print.
Purpose The purpose of this paper is to describe the evolving IFLA approach to impact evaluation through three of its international programmes: Freedom of Access to Information, Building Strong Library Associations (BSLA) and the International Advocacy Programme (IAP). This review positions these three programmes within the wider discourse of the international evaluation community. Design/methodology/approach Each of the three programmes is considered in turn to show what they were trying to achieve and how thinking about impact evaluation at IFLA is evolving. Findings This paper reports key evaluation findings for relevant phases of the BSLA and IAP programmes in general terms. Research limitations/implications The views presented are those of the evaluation consultants who advised each of these programmes (and in the cases of BSLA and the IAP conducted the programme evaluations). Practical implications The processes described and the conclusions drawn should be of interest to anyone involved in international or national library evaluation, especially of public libraries, library associations and national libraries. Social implications The paper suggests that more systematic impact evaluation of public libraries, library associations and national libraries is necessary to ensure their future survival. Originality/value The authors were uniquely placed to see and participate in IFLA impact evaluation discussions over the past decade.
✇ Performance Measurement & Metrics

Evaluating library service quality of University of Kashmir: a LibQUAL+ survey

7 de Maio de 2019, 10:49
Performance Measurement and Metrics, Volume 20, Issue 1, Page 60-71, February 2019.
Purpose The purpose of this paper is to evaluate the library service quality (LSQ) of University of Kashmir from the users’ perspectives based on the LibQUAL+ model approach. Design/methodology/approach The survey used the LibQUAL+ model to evaluate service quality (SQ) of Allama Iqbal Library (AIL), University of Kashmir, Srinagar (India). The data were collected from library users by administering the LibQUAL+ questionnaire in a printed format. Findings It has been revealed that AIL has succeeded in satisfying the minimum expectations of its users but could not meet the customers’ desired expectations that make them delighted. Library as Place (LP) and Information Control (IC) are the most satisfied dimensions of SQ of AIL, whereas Affect of Service (AS) is the least satisfied dimension. LP is the most desired dimension amongst LibQUAL+ dimensions. All LibQUAL+ items were found to be lying inside the zone of tolerance (ZoT). Further, “Library is secure and peaceful place for study, learning and research” (LP-21) was found as most important service item as per users’ desired expectations, while “Library staff shows dependability in handling users’ service problems” (AS-9) is marked as the least desired item. Originality/value Only a few studies have been carried out to evaluate the SQ of Indian university libraries as far as the LibQUAL+ model is concerned. Therefore, the findings of present survey will help to understand how far library has reached in satisfying users’ expectations as well as the areas or dimensions where it needs to improve.
✇ Performance Measurement & Metrics

Revision of an academic library user experience survey

7 de Maio de 2019, 10:49
Performance Measurement and Metrics, Volume 20, Issue 1, Page 48-59, February 2019.
Purpose The purpose of this paper is to explain how an academic library’s Student Experience Survey was revised and improved based on available research on survey design methodology and piloting efforts. Thus, it aims to contribute existing literature on library assessment and survey methodology by addressing issues of planning and coordinating assessment. The paper also provides a guideline on the survey revision process for existing instruments. Design/methodology/approach The library assessment coordinator collaborated with Assessment Coordinator Advisory Committee (AC2) to revise and improve the survey. The revision process started with adding suggested items and constructing new items based on library assessment planning survey administered to library faculty previously. Regular monthly meetings were held with the committee to facilitate the revision process. Multiple facets of the survey implementation were revised including wording, content, order and format of the items, response scales and survey distribution. Each facet was addressed based on available research on survey methodology and cross-checked with the committee members. Findings The revised 2018 University of Illinois at Chicago (UIC) Libraries Student Experience Survey consists of a total of nine items with response matrices, single choice response format and open-ended response format to measure students’ satisfaction, need and their use of the various library resources and services. The changes made to the survey are grouped into three categories: wording and content, response categories and scaling and design, font and layout. The revision process of the UIC Libraries Student Experience Survey can serve as an example for academic or other types of libraries that wish to track and report their impact regularly. Originality/value The UIC Libraries Student Experience Survey provides evidence that careful revision and piloting as well as obtaining relevant stakeholder buy-in to the process lead to build effective survey tools.
✇ Performance Measurement & Metrics

Usage, captures, mentions, social media and citations of LIS highly cited papers: an altmetrics study

28 de Fevereiro de 2019, 01:07
Performance Measurement and Metrics, Volume 20, Issue 1, Page 37-47, February 2019.
Purpose The purpose of this paper is to investigate the usage, captures, mentions, social media and citations of highly cited papers of Library and information science (LIS). Design/methodology/approach This study is quantitative research that was conducted using scientometrics and altmetrics indicators. The research sample consists of LIS classic papers. The papers contain highly cited papers of LIS that are introduced by Google Scholar. The research data have been gathered from Google Scholar, Scopus and Plum Analytics Categories. The data analysis has been done by Excel and SPSS applications. Findings The data indicate that among the highly cited articles of LIS, the highest score regarding the usage, captures, mentions and social media and the most abundance of citations belong to “Citation advantage of open access articles” and “Usage patterns of collaborative tagging systems.” Based on the results of Spearman statistical tests, there is a positive significant correlation between Google Scholar Citations and all studied indicators. However, only the correlation between Google Scholar Citations with capture metrics (p-value = 0.047) and citation metrics (p-value = 0.0001) was statistically significant. Originality/value Altmetrics indicators can be used as complement traditional indicators of Scientometrics to study the impact of papers. Therefore, the Altmetrics knowledge of LIS researchers and experts and practicing new studies in this field will be very important.
✇ Performance Measurement & Metrics

Information science’s contributions towards emerging open evaluation practices

30 de Novembro de 2018, 02:39
Performance Measurement and Metrics, Volume 20, Issue 1, Page 2-16, February 2019.
Purpose The purpose of this paper is to discuss emerging practices in open evaluation, namely, the concept of co-evaluation and how research on evaluation developed within information science can contribute to enhance stakeholders and citizens’ involvement in open science. Design/methodology/approach A meta-evaluative and transdisciplinary approach – directed toward the intersection between information science, evaluation, competences management, sustainability transitions management and participatory methodologies – provided the basis for the identification and subsequent reflection on the levels of stakeholder participation embedded into ISO 16439’s (2014) methods for assessing the impact of libraries and on the domains and competences to be mobilized for (co)evaluation. The contributions of Engaged 2020 Action Catalogue, as well as several taxonomies of evaluator competences and the Council of Europe’s (2016) conceptual model of competences for a democratic culture were particularly relevant for this (re)construction process. Findings Two results of the line of research carried out since 2012 at the Faculty of Social Sciences and Humanities of the Universidade NOVA de Lisboa (Portugal) can significantly contribute to improve stakeholders’ participation in Open Science: ISO 16439’s systematization of methods and procedures for assessing the impact of libraries and the (co-)evaluation competency framework. Originality/value This paper presents the transdisciplinary concept of co-evaluation and examines the current epistemological challenges to science by analyzing the general tendency to openness through the lens of research on evaluation and participatory methods developed within information science.
✇ Performance Measurement & Metrics

Bibliometric assessment of the global research output in Jatropha curcas Linn as reflected by papers indexed in Science Citation Index-Expanded

29 de Novembro de 2018, 11:57
Performance Measurement and Metrics, Volume 20, Issue 1, Page 17-26, February 2019.
Purpose The purpose of this paper is to examine the quantum of research papers and the citations these papers received for the plant Jatropha curcas Linn. Design/methodology/approach Articles published on Jatropha curcas Linn during 1987–2016 were downloaded from Science Citation Index-Expanded (SCIE) by using the keyword Jatropha* on October 18, 2017. The search resulted in 4,276 records in all. The authors analyzed only 4,111 documents which were published as review articles, research articles and proceeding papers using the complete count methodology. The data were analyzed to examine the pattern of growth of output, most prolific countries, institutions and authors. It also identified highly cited authors and journals used for communicating research results. Findings The study indicates that India, China and Brazil are the main contributors to the field and the pattern of growth indicates a steep rise in publication output especially in the last block of 2015–2016. Most of the prolific institutions and authors were also located in these countries. However, the impact of output was different from the pattern of output. The publication output is scattered in more than 1,000 journals published from different parts of the globe. Originality/value The plant of Jatropha curcas Linn is a highly useful plant as a source of biofuel energy. This is the second study in English language on this plant and has used a large set of publication data as compared to the first. The findings of the study may be useful for policy makers as well as for researchers working in the field of biofuel energy.
✇ Performance Measurement & Metrics

Investigating first-generation students’ perceptions of library personnel

18 de Outubro de 2018, 09:50
Performance Measurement and Metrics, Volume 20, Issue 1, Page 27-36, February 2019.
Purpose The purpose of this paper is to investigate the perceived role of library personnel in supporting first-generation students at Penn State University Libraries, and also how students’ perceptions of library personnel change over time, and the various experiences that influenced their changes in perception. Design/methodology/approach This study employed focus groups to solicit input from first-generation students. A four-step team-based approach to qualitative coding process was developed including the development of a codebook informed by common themes and concepts drawn from the literature. Findings Findings indicate that operating from a deficit of library cultural capital often results in low awareness of available services and changes in perception are more influenced by personal exploration than limited interactions with personnel. Further, while currently employed interventions are well targeted, opportunities exist for enhancing efforts. Research limitations/implications As this is a case study, the findings are not generalizable. Per conducting only four focus groups, the experiences of participants may not represent the typical scope of personnel-related interactions. Originality/value This study adds to the limited body of evidence that first-generation students’ struggle from a deficit of library-related cultural capital.
✇ Performance Measurement & Metrics

Barriers and solutions to assessing digital library reuse: preliminary findings

4 de Outubro de 2018, 02:54
Performance Measurement and Metrics, Volume 19, Issue 3, Page 130-141, November 2018.
Purpose The purpose of this paper is to highlight the initial top-level findings of a year-long comprehensive needs assessment, conducted with the digital library community, to reveal reuse assessment practices and requirements for digital assets held by cultural heritage and research organizations. The type of assessment examined is in contrast to traditional library analytics, and does not focus on access statistics, but rather on how users utilize and transform unique materials from digital collections. Design/methodology/approach This paper takes a variety of investigative approaches to explore the current landscape, and future needs, of digital library reuse assessment. This includes the development and analysis of pre- and post-study surveys, in-person and virtual focus group sessions, a literature review, and the incorporation of community and advisory board feedback. Findings The digital library community is searching for ways to better understand how materials are reused and repurposed. This paper shares the initial quantitative and qualitative analysis and results of a community needs assessment conducted in 2017 and 2018 that illuminates the current and hoped for landscape of digital library reuse assessment, its strengths, weaknesses and community applications. Originality/value In so far as the authors are aware, this is the first paper to examine with a broad lens the reuse assessment needs of the digital library community. The preliminary analysis and initial findings have not been previously published.
✇ Performance Measurement & Metrics

A meta-analysis of service quality of Iranian university libraries based on the LibQUAL model

26 de Setembro de 2018, 07:31
Performance Measurement and Metrics, Volume 19, Issue 3, Page 186-202, November 2018.
Purpose The purpose of this paper is to assess the quality of Iranian university libraries. Design/methodology/approach This first systematic review and meta-analysis were based on the PRISMA guidelines by searching in national and international databases from 2003 to January 2017 with standard Persian and English keywords. Data searching, extracting and quality appraising were completed by two researchers, independently. Any unexpected documents were assessed by a third expert researcher. Data were extracted in accordance with the “Strength of the Reporting of Observational Studies in Epidemiology” checklist after the final selection of appraised documents. Random effects size based on Cochrane test and I2 were used for combining the obtained results from different studies together by considering the heterogeneity of studies. Findings Based on the meta-analysis conducted in 25 (6.42 percent) included studies, the total sample size was estimated. According to three dimensions of LibQUAL, findings of current information control, affect of service and the library as a place were estimated as 5.37 [CI95%: 5.02, 5.73], 6.91 [CI95%: 5.56, 6.26], and 5.46 percent [CI95%: 5.2, 5.73], respectively. Also, mean of service adequacy and superiority gap are equal to 0.07 [CI95%: −0.22, 0.36] and −2.06 [CI95%: −2.89, −1.23], respectively. There was a significant correlation between three dimensions of service quality and service superiority gap of LibQUAL and geographical regions of Iran (p<0.01). Also, a significant correlation was found between the gaps of services and three aspects of LibQUAL model and published years through a meta-regression test (p<0.01). Practical implications The results obtained from the present study showed that users are relatively satisfied with the quality of services provided by Iranian university libraries. An improvement in the quality of library services can promote the scientific level of universities. Originality/value The results of the present systematic review and meta-analysis study demonstrate a vital connection between primary research studies and decision-making for policymakers in Iranian university libraries to increase quality services.
✇ Performance Measurement & Metrics

International STEM graduate students: reaching a deeper understanding

19 de Setembro de 2018, 10:40
Performance Measurement and Metrics, Volume 19, Issue 3, Page 153-165, November 2018.
Purpose The purpose of this paper is to better understand the perceptions international STEM students have of the library and higher education based on their responses to the Ithaka S+R Graduate Student Survey. Design/methodology/approach To better understand these groups, this study conducted the Mann–Whitney–Wilcoxon test on the Ithaka survey results to compare the groups and identify any statistically significant differences that the international STEM graduate students group (ISG) has to other groups. Findings This paper found that ISG valued Higher Education objective variables more than the non-ISG group, with the exception of one question. The ISG group also valued 7 of the 13 role of the library (ROL) variables statistically different. Research limitations/implications Since the students self-reported as international or STEM, the authors are unable to assess whether the response pool is representative of the university as a whole. Practical implications By understanding how international/STEM students may differ from other populations, libraries can better design spaces and services for these groups. Originality/value Existing studies tend to focus on international students or STEM students and information literacy. This study intends to fill a gap in the understanding of how these groups perceive the ROL and their education.
✇ Performance Measurement & Metrics

Input resources indicators in use for accreditation purpose of higher education institutions

23 de Agosto de 2018, 12:11
Performance Measurement and Metrics, Volume 19, Issue 3, Page 176-185, November 2018.
Purpose Higher education is a complex system that involves multiple inputs and outputs, where various activities and processes are performed. The purpose of this paper is to monitor the input resources used for executing various activities of higher education institutions. These resources are classified into three types: human resources, physical resources and financial resources. Design/methodology/approach The author examines various national and international accreditation standards to determine their requirements for key performance indicators (KPIs) to monitor input resources. Moreover, the author uses implications proposed by previous research and best practices. Findings A set of appropriate and generic KPIs is developed for each type of these resources leading to a total of 72 key indicators. These indicators are easy to measure, which makes them practical to be adopted by a large scale of institutions. Practical implications The proposed indicators provide adequate information to administrators and policy-makers, accrediting bodies and stakeholders to identify the progress and achievements. These indicators are also used for benchmarking purposes by comparing the institution’s performance against their comparable institutions counterparts. Moreover, they are used for marketing purposes to commercialize the institution by attracting prospective students and teaching faculty in addition to increasing current students’ satisfaction. Originality/value The paper gives special attention to developing a set of generic KPIs for assessing the availability and quality of input resources used for carrying out various activities of higher education institutions for the aim of improving their performance and hence helping them comply with the requirements of accreditation standards.
✇ Performance Measurement & Metrics

Reading, writing, and … running? Assessing active space in libraries

21 de Agosto de 2018, 10:25
Performance Measurement and Metrics, Volume 19, Issue 3, Page 166-175, November 2018.
Purpose The University of Tennessee Knoxville’s Libraries has set aside space and stocked it with treadmill desks, standing desks, cycling desks and balance chairs to encourage physical activity while using library space to promote active learning. The purpose of this paper is to assess the impact of this innovative space on library users through a study conducted by a research team using observations and short surveys to gather information about usage trends and user perceptions of this “active learning space.” Design/methodology/approach This study used both ethnographish observation and self-selected survey. Researcher observation notes were used to gather usage rates of the space and equipment in the space, and survey responses were coded for themes to identify user perceptions around the space. Findings The findings strongly suggest that users find mental and physical health value in the “active learning space” and many would find value in the expansion and improvement of the space. Research limitations/implications The limitations of this study include a shorter observation period compared to the survey collection period and limited demographic collection to shorten the survey instrument. Originality/value However, this study was able to assess how an active learning space in an academic library can influence and have a significant impact on student success.
✇ Performance Measurement & Metrics

How assessment websites of academic libraries convey information and show value

10 de Agosto de 2018, 03:11
Performance Measurement and Metrics, Volume 19, Issue 3, Page 203-212, November 2018.
Purpose As libraries are required to become more accountable and demonstrate that they are meeting performance metrics, an assessment website can be a means for providing data for evidence-based decision making and an important indicator of how a library interacts with its constituents. The purpose of this paper is to share the results of a review of websites of academic libraries from four countries, including the UK, Canada, Australia and the USA. Design/methodology/approach The academic library websites included in the sample were selected from the Canadian Association of Research Libraries, Research Libraries of the United Kingdom, Council of Australian University Libraries, Historically Black College & Universities Library Alliance, Association of Research Libraries and American Indian Higher Education Consortium. The websites were evaluated according to the absence or presence of nine predetermined characteristics related to assessment. Findings It was discovered that “one size does not fit all” and found several innovative ways institutions are listening to their constituents and making improvements to help users succeed in their academic studies, research and creative endeavors. Research limitations/implications Only a sample of academic libraries from each of the four countries were analyzed. Additionally, some of the academic libraries were using password protected intranets unavailable for public access. The influences of institutional history and country-specific practices also became compelling factors during the analysis. Originality/value This paper seeks to broaden the factors for what is thought of as academic library assessment with the addition of qualitative and contextual considerations.
✇ Performance Measurement & Metrics

Comparison of personas between two academic libraries

8 de Agosto de 2018, 08:52
Performance Measurement and Metrics, Volume 19, Issue 3, Page 142-152, November 2018.
Purpose A persona describes a group of library patrons as a single person to better identify and describe user patterns and needs. Identifying personas in academic libraries can assist in library planning by focusing on patrons. Initially, personas were thought to be unique to each library; additional insights led the researchers to rethink this assertion. The purpose of this paper is to determine if personas, developed in one library, are unique or more universal than previously thought. Design/methodology/approach In this study, 903 surveys were completed across two institutions asking library patrons to identify use patterns within each library. Mean score responses were analyzed using an ANOVA, principal component analysis and RapidMiner technology. All analyses were used to identify personas with common interests and places personas in groups or neighborhoods. Findings The findings provide evidence for the universality of academic library personas. However, differences occur in how the personas are grouped and use different library services and resources. Originality/value Personas allow librarians to view patrons in a more personal way as they connect personas to specific library spaces. While the personas appear to be universal, their interactions with each other depend on specific library amenities.
✇ Performance Measurement & Metrics

Demonstrating the value of “library as place” with the MISO Survey

1 de Junho de 2018, 08:28
Performance Measurement and Metrics, Volume 19, Issue 2, Page 111-120, July 2018.
Purpose The purpose of this paper is twofold. It first examines cross-institutional benchmark data about “library as place” from 99 US schools in the Measuring Information Service Outcomes (MISO) Survey ( The data demonstrate the value of “library as place” to students in particular. Second, the paper shares case studies of how two college libraries made MISO Survey “library as place” data actionable. Lafayette College (Easton, Pennsylvania) analyzed local MISO Survey data after a renovation to validate return on investment. Earlham College (Richmond, Indiana) analyzed MISO Survey data to help secure a science library renovation and to justify an architectural study for its main library. Design/methodology/approach The paper begins with an analysis of “library as place” using aggregate benchmarks derived from US college and university respondents between 2012 and 2015. Specifically, the paper contrasts student and faculty perceptions of “library as place” via national benchmarks about: library services importance, satisfaction, and use (three benchmarks); hybrid online/“place-based” library services importance, satisfaction, and use (three benchmarks). Pivoting from higher education to individual, local perspectives, two case studies reveal how academic libraries used MISO Survey findings to demonstrate the value of “library as place” for renovation purposes. Findings The findings include that undergraduates make more frequent use than faculty of place-based services such as reference, equipment loans, and physical course reserves. Undergraduates also find most of these services more important than faculty do. Faculty makes generally more frequent use than undergraduates of online services such as library databases and the catalog. They find that these services to be more important than undergraduates do. Faculty and undergraduates use newer library discovery systems with equal frequency and find them to be equally important. Undergraduates find comfortable library spaces to be very important, and faculty considers them to be only a bit less important. Originality/value This is the first paper using MISO Survey data to focus on the importance and satisfaction of place-based library services involving cross-institutional comparisons for students and faculty. Previously published research using MISO Survey data have compared the use of place-based library services. This is also the first paper to offer case studies about how institutions use MISO Survey data to demonstrate the value of “library as place.”
✇ Performance Measurement & Metrics

Assessment’s role in building student support to facilitate change

30 de Maio de 2018, 08:29
Performance Measurement and Metrics, Volume 19, Issue 2, Page 101-110, July 2018.
Purpose The California State University Library successfully improved its library’s hours, including introducing 24-hour access, thanks in part to an assessment process that helped the University Library to develop a partnership with the students. The paper aims to discuss this issue. Design/methodology/approach In order to understand the need for expanded operational hours, students’ were surveyed on their satisfaction with current library hours, suggestions on how best to meet their actual hour needs, and interest in 24-hour access. Efforts to solicit student input included involving the student government in promoting the survey and reviewing its results, a decision that later had a major impact on the success of the project. Findings The survey identified days with problematic hours and which hours student suggested changes to that would best fit most students’ needs. In response to the findings, the library immediately implemented incremental changes that fit within current library resources. The survey’s most significant finding was that 94 percent of students wanted 24-hour study in the library. Originality/value An unexpected benefit of the library making immediate, incremental changes was that students could see the results of their input and feel a sense of ownership. The findings also led to the library making 24-hour access a top priority. When university funding was not available, the university’s student government approached the library with a unique proposal to include funding for 24-hour library access in their proposed student fee increase. The student body showed its willingness to invest in the library by passing the fee increase. California State University San Marco’s library began 24/5 hours in Fall 2017.
✇ Performance Measurement & Metrics

Quality in museums and libraries: a comparison of indicators

14 de Maio de 2018, 09:37
Performance Measurement and Metrics, Volume 19, Issue 2, Page 90-100, July 2018.
Purpose The purpose of this paper is to define quality criteria for museum work and try to identify indicators for assessing such quality, using the well-known quality indicators for libraries as basis of comparison. Design/methodology/approach In 2016, the first international standard for museum statistics was published (ISO 18461). At the same time, the museum community realized the need to proceed from statistics to quality assessment in order to prove the effectiveness and efficiency of museum performance and finally also the influence of museums on society. The paper analyses the existing literature on museum quality and museum evaluation and compares the findings to the established methods in the library sector. Findings Comparison of quality criteria and quality evaluation in museums and libraries shows not only many similarities in what is regarded as quality, but also considerable differences in the options for assessment. Research limitations/implications The paper shows a tentative selection of potential quality indicators and quality assessment methods for museums. Further work and practical expertise will be needed for developing and testing a reliable set of methods. The new ISO project 21246 key indicators for museums will further this development. Practical implications There is an evident similarity in the mission and goals of cultural institutions such as archives, libraries and museums. Methods already in use in libraries for assessing performance quality and library impact can therefore also be useful in the museum sector. But the immense variety of museums requires an adaptation of existing methods and a number of new indicators especially devised for museum tasks. Originality/value The paper describes possible solutions for evaluating the quality of museums. As quality criteria are similar in museums and libraries, new quality indicators for museums may in turn become useful in the library sector.
✇ Performance Measurement & Metrics

An agile planning and operations framework

3 de Abril de 2018, 10:36
Performance Measurement and Metrics, Volume 19, Issue 2, Page 121-126, July 2018.
Purpose The purpose of this paper is to describe the design and implementation of an agile planning and operations framework for an academic research library, designed to facilitate an ongoing organizational conversation about the organization’s strategic intent and how it plans to move from intention to reality. The goals motivating the implementation of such a framework include creating iterative, open-ended planning and management processes that enable increased flexibility and openness to unforeseen opportunities, as well as the ongoing integration of library faculty, staff, and external stakeholder voices into planning, management, and assessment discussions. Design/methodology/approach The framework seeks to harmonize planning, management, and assessment processes over 18-24-month rolling time horizons, during which organizational efforts and investments would be reviewed and revised in an iterative fashion. Findings The authors share results and analyses from stakeholder assessments used to develop foundational mission, vision, values, and strategic directions documentation and discuss the structural, cultural, and organizational development challenges confronted and gains experienced in implementing the framework. Originality/value Many academic libraries are exploring new approaches to strategic planning, ways to enhance organizational health, and manage change. The authors are unaware of an academic or research library that has attempted to design and implement a similar approach to strategic planning and its assessment. The agile planning framework provides an alternative to traditional “waterfall” approaches to strategic planning for libraries.
✇ Performance Measurement & Metrics

Understanding students’ satisfaction with OERs as course materials

18 de Janeiro de 2018, 09:59
Performance Measurement and Metrics, Volume 19, Issue 1, Page 66-74, February 2018.
Purpose The purpose of this paper is to introduce a survey instrument to measure three components of students’ perceptions of open and affordable course materials – quality, integration, and experience – and discuss its reliability and predictive validity. Design/methodology/approach The authors distributed an end-of-semester online survey to students enrolled in sections of 12 courses that adopted OER in Fall 2016, as well as conducting a within-interview survey with the instructors of those courses. The authors calculated the descriptive statistics from the responses to the student survey, as well as examining the inter-item and inter-rater reliability of the instrument. Finally, explored correlations in the data gathered through both the student and faculty surveys were explored. Findings The authors found that both students and faculty were generally pleased with the quality and experience of using open and affordable digital materials. The authors also found that our three survey subscales had strong inter-item reliability, and that the quality and experience subscales had predictive validity in terms of whether students would choose a traditional or digital text in future courses. Originality/value In addition to providing evidence in terms of the full survey instrument’s reliability and predictive validity, factor analysis indicates that a short scale of quality and experience Likert scale items could be used by practitioners to effectively assess satisfaction of digital materials among traditionally aged undergraduate students.
✇ Performance Measurement & Metrics

Experience assessment: designing an innovative curriculum for assessment and UX professionals

18 de Janeiro de 2018, 09:59
Performance Measurement and Metrics, Volume 19, Issue 1, Page 30-39, February 2018.
Purpose While assessment and user experience (UX) have been identified as areas of growing focus in all types of libraries, there is currently little infrastructure to prepare students for these roles (Applegate, 2016; Askew and Theodore-Shusta, 2013; Nitecki et al., 2015; Oakleaf, 2013; Passonneau and Erickson, 2014). As a step toward addressing this gap, a team from an American Library Association-accredited master’s program situated at a large public land-grant institution (LGU) worked with practitioner partners from academic libraries and information agencies to develop a new model for preparing information professionals with assessment and UX expertise. The paper aims to discuss these issues. Design/methodology/approach In fall of 2015, faculty members applied for funding from the US Institute for Museum and Library Services Laura Bush 21st Century Librarians program for a program to develop formalized assessment and UX training in Library and Information Science (LIS) education. The student cohort would have interests in two areas: academic libraries and specialized information agencies. The two groups would complete much of the same coursework, earn the ALA-accredited master’s degree and have the opportunity to engage in co-curricular activities focused on UX and assessment. However, each sub-group would also pursue a subject-specific curriculum. In April 2016, IMLS funded the program. Findings In addition to reviewing the literature related to best practices in curriculum development, the authors describe the process of designing the program, including the curriculum, co-curricular mentoring and practicum opportunities, and the tools developed to evaluate the program’s effectiveness. Research limitations/implications At a time in which the library practitioner and LIS educator communities are contemplating how best to prepare professionals with much-needed expertise in assessment and UX, UX-A represents an innovative approach in professional preparation. Although the UX-A program is grant-funded, several of the program components could be adapted and incorporated without such support. Originality/value This paper discusses the structure and history of the program, issues related to developing a new curricular program for LIS education, and the educational and professional development needs of the assessment and UX professional community. It includes an extensive review of literature related to LIS curriculum development, practica, and professional mentoring, as well as suggestions for implementing elements of the program in other settings.
✇ Performance Measurement & Metrics

The use of academic libraries in turbulent times

18 de Janeiro de 2018, 09:59
Performance Measurement and Metrics, Volume 19, Issue 1, Page 40-52, February 2018.
Purpose The purpose of this paper is to explore how an innovation in the University Management Information System was leveraged to incorporate library data by an initially sceptical strategic management team. The rationale was to extract evidence of correlations between library use and student achievement. This kind of information is of particular interest to the institution, which is at present dealing with crises popularly summarised in the slogan “#FeesMustFall” among students who suffer from the effects of poverty and exclusion in higher education. Comment is offered on some of the relationships between student library behaviour before, during and after the nationwide disruptions that destabilised universities and threatened their survival at the end of 2016, just before the final examination period. Design/methodology/approach Data were extracted from the data warehouse from the comparative demographic perspectives of students’ degrees of disadvantage in an effort to uncover any hitherto hidden patterns of library use. Findings The use of the library as expressed by footfall and loans was mapped against students’ pass rates and their collective GPA, indicating that increased library use correlates positively with better academic performance. Some of the initial correlations between student library behaviour before, during and after the nationwide disruptions that destabilised universities and threatened their survival at the end of 2016 just before the final examination period are explored. The effects that library closures (under threat of damage) at a critical time in the academic year might have had on library use and on student performance are interrogated. Practical implications Students on financial aid, which was used as an indicator of disadvantage, come from schools and environments where access to information technology and libraries is very limited, so that library habits are either poorly established or not at all. At the University of Cape Town (UCT), considerable support is in place for students to encourage the development of library habits. An analysis of available data indicates that students who have acquired library habits regardless of unfavourable financial circumstances do not exhibit behaviour and academic outcomes markedly different from that of their more privileged peers. Originality/value Combining library data with data from the university data warehouse is a new approach in South Africa. It is an approach that is of value both to the library and the institution at large and has brought meaningful insights into the role the academic library might be seen to play in promoting student academic achievement.
✇ Performance Measurement & Metrics

Articulating value and impact through outcome-centered service delivery

18 de Janeiro de 2018, 09:59
Performance Measurement and Metrics, Volume 19, Issue 1, Page 2-11, February 2018.
Purpose The purpose of this paper is to share the ways in which student and learning support at the University of Sunderland has embedded and matured a new outcome-centered performance model – the Quality Model – in order to create an agile evidence-base of value, outcome and impact evidence. The authors will also share how, having established the fundamental principles regarding value and impact capture in our library setting, the concepts and approaches have also been developed and applied successfully within the context of multi-converged service delivery across the wider student and learning support service, using the AMOSSHE Value and Impact Toolkit. Design/methodology/approach The authors’ approach will be illustrated with two case studies, the first focusing on the university library services and the second on the student counseling service. Findings The findings will reveal that by establishing an outcome-focused model, the authors have been able to apply it across a converged service in order to generate the evidence required to articulate the value and impact of our key service objectives. Originality/value As a performance approach, the Quality Model is an original concept in that it is a self-formed model designed to meet the strategic needs of the University of Sunderland. It differs from many performance models in that it is founded on a holistic approach to service culture and customer-relationship management and is based upon strategic marketing principles. The AMOSSHE Toolkit is a pre-existing toolkit which is fully referenced in the paper. The Quality Model and AMOSSHE Toolkit are of particular relevance as many higher education services are increasingly challenged to demonstrate their value and impact and the outcomes their services deliver. This calls for a strategic approach to managing qualitative evidence. Therefore, although bespoke, the approach is transferable to the strategic priorities of other HE settings.
✇ Performance Measurement & Metrics

Academic libraries and student learning outcomes

18 de Janeiro de 2018, 09:59
Performance Measurement and Metrics, Volume 19, Issue 1, Page 75-87, February 2018.
Purpose The purpose of this paper is to examine the literature on student learning outcomes and academic libraries. An analysis of 81 papers revealed the following themes: outcomes assessment of library skills instruction, tools for assessing students’ achievement of learning outcomes, institutional accreditation and student learning outcomes, academic libraries’ impact on student learning outcomes, and the creation of learning outcomes. An understanding of these themes highlights how student learning outcomes are created, assessed, and valued in the academy. Design/methodology/approach The methodology centered on a literature review of student learning outcomes and academic libraries from 2001 to the present. To identify relevant articles on this topic, the authors searched the library literature with various combinations of the following keywords: learning outcomes, library, academic, university, college, and higher education. Findings An analysis of 81 papers on student learning outcomes and academic libraries revealed the following five themes: outcomes assessment of library skills instruction, assessment tools for student learning outcomes, institutional accreditation and student learning outcomes, academic libraries’ impact on student learning outcomes, and the creation of learning outcomes. Research limitations/implications The research was limited to articles published in English and after 2000. Practical implications The findings can be utilized to inform instructional librarians about creating and assessing student learning outcomes. In addition, the paper can assist librarians in forging partnerships with academic departments and faculty in creating student learning outcomes to support course and departmental outcomes and accreditation efforts. Originality/value The research offers librarians opportunities to incorporate student learning outcomes in library instruction as well as collaborate with faculty in creating student learning outcomes.
✇ Performance Measurement & Metrics

Contextualizing library assessment within a broader ecosystem

18 de Janeiro de 2018, 09:59
Performance Measurement and Metrics, Volume 19, Issue 1, Page 18-29, February 2018.
Purpose The purpose of this paper is to draw a picture of the current landscape of library assessment based on the data gathered in interviews. The authors will focus specifically on the continuum between micro and strategic assessment and share the lessons learned from diverse institutions and geographies about how to build a culture of assessment. Design/methodology/approach Between 2015 and 2017, the researchers have interviewed more than 75 library directors and leaders, library assessment practitioners, and academic experts on four continents about library assessment and its current state in their institutions. Findings The results reveal a varied landscape, with libraries in widely varying stages of assessment performance and readiness. Originality/value This paper presents the results of a large-scale study over more than two years. More than 75 people have been interviewed in five countries. The scale and scope of the work is both significant and unique.