Quality and Staff Development Coordinator

Northern Territory University

Co-ordinator (Library Staff Development and Quality Programs)

University of New South Wales

for the

Evaluations and Investigations Program (EIP) team

which also included

Manager

University of Ballarat Information Literacy and Research Services

and

Director, Library Services

Queensland University of Technology Library

This paper summarises best practice activities in Australian university libraries now and suggest ways in which these may develop in the near future. It is based on information from a recent project funded by the Australian commonwealth department with oversight of higher education.

To understand the reasons for this national project and to provide background to the information drawn together in it, it is useful to briefly look at some characteristics which shape the self perceptions of Australian university libra! ries, the perceptions individual institutions have of their Australian peers, and the perceptions of university libraries in relation to the wider world.

Australian university librarians make particular efforts to overcome the considerable limitations of the "tyranny of distance" to represent their libraries at regular meetings. Taking a national approach to issues is seen as important.

Distant globally, Australian university libraries look particularly to the US and the UK for examples of innovation: but, there is also a definite Australian tendency to do-it-yourself action.

Smaller, newer regional universities and their libraries have been nimble and focussed on niche markets, often motivated to overcome relative disadvantage by exploiting the benefits of technology.

With an ever more ! diverse institutional population, including substantial numbers of full fee paying international students, maintaining true user focus is a complex process.

There were six prewar Australian universities. There are now thirty eight. Some perceptions about hierarchies of age, size and research levels do exist but the divisions are not entrenched, and all libraries are open to the adoption of good practice.

Australian academic libraries are actively involved in the implementation of quality frameworks and are utilising quality management tools such as benchmarking and performance measurement. These activities are not necessarily well known or communicated outside the originating institution. Time factors are ever more critical and there is a certain perception that some things may not be worth the effort. There are obvious benefits in a greater sharing of experience and ideas, part! icularly for smaller, regional and often less generously funded institutions.

The Council of Australian University Librarians has been concerned to facilitate access by Australian university libraries to information to assist them with the implementation of best practice initiatives. This EIP project is one of the few such projects to examine library issues in Australian higher education over the past decade.

A number of key issues were addressed, including the investigation of:

An early project challenge was achieving consensus on terminology. To date, there appear to be no universally accepted definitions for best practice, benchmarking and performance measurement. The literature supports the difficulty of finding agreement on terms, stresses the importance of using endorsed definitions and discusses the discomfort around the use of the terminology in many university libraries, particularly when first encountered or initiated. Libraries found terminology far less of an issue as use made the words part of the culture.

Through a combination of surveys, site visits to prospective 'interesting' sites and through contacts and an extensive review of Australian and international literature on the topic, the project team was able to! investigate and evaluate the current situation of university libraries.

The references included in the report and as background material in a proposed handbook were gathered from databases, the internet, contacts with colleagues, local and international visits, discussion lists and other sources.

In view of the potentially large volume of material, it was decided the emphasis would be on providing insight into the current level and type of activity within academic and research libraries and information services. Sources which provided insights into relatively recent work were candidates for the review section and, if particularly significant, for the substantial annotated useful sources listing. Most material, with the exception of some key resources, was published within the last five years. Sources for the investigation of more historical material were identified for the dedicated.

Three s! urveys were developed and distributed to all Australian (38) and New Zealand (6) university libraries and the major Australian non academic research libraries (State Libraries, National Library of Australia, and CSIRO, the scientific research libraries network). These aimed to identify best practice activities, past, present and planned.

Site visits by members of the project team were undertaken to those libraries identified as exemplars of interesting initiatives. These allowed further exploration and clarification of issues raised in the survey findings and enabled the documentation of best practice activities within these libraries. This information may be transferable to other sites or appropriate for inclusion in the proposed handbook for busy practitioners.

As part of an unrelated institutional initiative, a member of the project team had undertaken visits to a number of international library and university sites in 1998. From t! his exercise information was fed into this project and provided data which mirrored and complemented the Australian investigations. Brief international case studies were included in the report since the use of standard questionnaires for international comparisons was not feasible in this situation.

Generally, there is a relative dearth of material published on local library experiences despite evidence of considerable activity from the project investigations. Relevant Australian library conference papers form a very small subset of current professional publications. Anecdotal evidence suggests that Australian librarians look to the business oriented quality bodies within Australia, such as the Australian Quality Council, for keeping up to date with the literature and to the international library literature. A number of libraries appeared to have "read the book" on ISO and ! IFLA, and recent work on electronic sources but there was little evidence of reference to other areas of the international literature.

Benchmarking is now regarded as a useful and appropriate tool for improving products, processes and services despite past perceptions of it as a tool for business / industry / profit oriented organisations. Almost half of Australian university libraries reported involvement in benchmarking exercises over the past few years.

The strong tradition of informal co-operative surveying and aggregation of data as demonstrated through instruments such as the CAUL surveys may have contributed to this acceptance. Most university librarians have access to round robin surveys on topics of widespread interest. These supplement the Australian equivalent of the ARL statistics (AARL statistics) now approaching their fiftieth year. These provide time series covering the standard quantitative indicators used in the past! as "benchmarks". Access to these is available through the CAUL web site.

A range of benchmarking methodologies have been used, choice depending on institutional goals and objectives, size and structure and the type of process identified for benchmarking.

Examples of in depth benchmarking of processes include:

Examples of using common survey instruments to extract comparable data with the aim of process improvement include:

Areas whi! ch have been benchmarked with organisations other than libraries include:

Australian university libraries are very open to approaching distant benchmarking partners. Partners within close regional groupings, whilst convenient, are perhaps seen as providing less challenge.

First steps in international partnering have been taken through membership of and participation in the ACU CHEMS Benchmarking Club and Universitas 21 by a small number of the larger libraries. International benchmarking presents particular challenges. The CHEMS approach is also currently being used "in collaborative" dialogue with the McKinnon Walker/ IDP project to develop national sets of outcome benchmarks in areas of university management including library and information services.

Informal benchmarking activities in! cluded such areas as:

Formal benchmarking for some was regarded as informal benchmarking by others.

Focus by large commercial organisations on broad industry metrics rather than more detailed process benchmarking is not as evident amongst Australian commercial organisations. The wider Australian benchmarking scene is dominated by the Australian Quality Council. Some university libraries are becoming more involved with AQC activities such as the AQC benchmarking networks, use of the Quality Awards for Business Excellence framework and the alignment of benchmarking information analysis with this framework. The framework is broadly similar to the European and Baldrige counterparts.! AQC benchmarking networks go to considerable expense to provide for detailed examination of the nominated processes. Network participants are involved in data gathering and analysis and comprehensive site visits. Some small commercial companies offering data analysis and questionnaire design services also use the Business Excellence framework so that benchmarking between libraries and between libraries and commercial organisations is becoming easier with the sharing of a common approach.

Australian universities appear to be less conscious of protecting the integrity of their institutions and retaining competitive advantage than in the US where less formal exchanges of views, cooperation on new initiatives or joint lobbying for a greater good are preferred to benchmarking

The British pattern of dominance of benchmarking and other initiatives by newer universities keen for a competitive advantage is not generally the case in Australia. The European patt! ern of using regional links to open up possibilities for more cooperative activities between academic and other libraries is also not typical in Australia. Australian benchmarking partners are usually a mixture of newer and older institutions, with many partnerships including libraries as far apart as it is possible to be on the Australian continent.

Cultural, political and historical barriers are not issues within Australia but are issues of great relevance in dealing with the international benchmarking that new alliances are bringing. As a sovereign nation with a tendency to national uniformity, Australia still has much to learn in the global context. Australian libraries in the Universitas 21 grouping and those involved in recent CHEMS library benchmarking are becoming more conscious of cultural and political sensitivities.

Performance measurement is universally practised as part of the management process in Aus! tralian university libraries. A wide range of performance indicators are being used including the three published CAUL indicators, in order of frequency of use:

Few libraries use all three indicators. A substantial minority use the materials availability indicator regularly and slightly fewer the client satisfaction indicator.

Some libraries have adapted and adopted a range of approaches tailored to need including Van House, Hernon and Altman and Parasuraman based instruments.

The improvement of CAUL indicators and national preferences for further development were investigated. The CAUL supported CAVAL (CRIG) Working Party on Performance Measures for Reference Services, has recently published a final report recommending twelve indicators in three broad dimensional groupings. The model is called ASK (attributes, support, knowledge).Some libraries expressed! interest in trialling a range of these recommended PI's.

Much effort has also been expended on the development of in-house indicators and the modification / adaptation of other externally published indicators. The over-riding motivation appears to be to develop indicators which relate closely to strategic plans and key result areas. Australia has been slower in the development of performance indicators for the electronic library despite some interesting speculative work in individual institutions. Some libraries mentioned following the work of McClure, Lopata and Brophy. There was no evidence of awareness of other EU sponsored projects.

The highest rated responses to priority indicator development were:

The Australasian Universitas 21 members are aligning the use of performance indicators a! nd evaluative surveys amongst members. Surveys are also being aligned more closely with the Australian Quality Awards framework so results can feed into benchmarking the wider business community.

There is a definite trend towards increased application of quality frameworks. The influence of the Commonwealth government sponsored Quality Audits in 1993-1995 has already been documented in an article by Vicki Williamson and Andy Exon. At that time the first TQM/TQS, Best Practice projects were being implemented in libraries. Although less than 50% of the respondents to the current surveys have a fully implemented framework, the majority indicated their intention to investigate, implement or further develop frameworks over the next year.

Two university libraries and one university have been awarded national recognition by the Australian Quality Council through the Quality Awards. This recognition has provided leadership for ! others.

Local influences such as the:

all played a part in encouraging use of quality principles.

Motivating factors for implementation include:

Formal frameworks currently in use include:

There were also local adaptations based on some of the above, the Scottish Quality Management System and various frameworks for vocatio! nal education and training. ISO accreditation is particularly associated with electronic services and IT activities.

There is European and UK interest in adapting the European Business Excellence Model framework for quality management purposes and British universities have also had the choice in recent years of a range of government endorsed quality frameworks such as the Chartermark. Apart from some short-lived curiosity about Iip, it is the use of the AQA framework which interests Australian academic libraries most. About one quarter of (mainly) larger Australian university libraries have, or intend to implement aspects of the AQA framework. Interest has grown from the first use in this framework over five years ago in one library. Australia is thus well in the forefront of a more recent international trend towards interest in adaptations of national quality award frameworks.

Whilst many Australian libraries have emb! raced quality improvement initiatives with enthusiasm, the training and competencies required for staff to work effectively with the tools and techniques has not always been addressed to the same degree. Libraries with ongoing training programs which address quality management techniques and related skills are rare: yet the presence of integrated in-house staff development programs appears to be one prerequisite for a really effective framework implementation in a large institution. Some respondents acknowledged that a lack of appropriate training at the appropriate time contributed to the less than satisfactory aspects of their implementation.

It is recognised that a significant investment of resources is required for successful staff development. Training support is more likely to be in place where the institution has dedicated staff with some specific responsibility for quality, staff development and training. Access to training tended to be more formal if the pr! ograms were university initiatives and if training was conducted by an external facilitator, or, more rarely, by a university training unit. Less appropriate training included the one off, 'sheep dip' approach which could be counterproductive.

Some training had been sourced from the US from training companies supplying this type of training to U.S industry or to the TQM/CQI training programs implemented in some U.S universities. AQC training programs for quality professionals and awareness and skills training for other staff were also used. The AQC has also developed nationally endorsed quality management competencies which form part of the government approved National Training Agenda.

Project results and recommendations will go to the November 1999 CAUL meeting in Auckland. The full EIP report should be published in 2000 as part of an ongoing series by the government publishing service for DETYA.

Sugge! stions for the future include:

Further developments should emerge early in 2000 after CAUL has considered the information.

The investigations: