Using assessment of student learning outcomes to measure university performance: towards a viable model

Abstract

This study investigates the possibility of developing a suite of performance indicators which could measure differences in universities’ performance in attainment by their students of specified institutional or course-based learning outcomes. The measurement of learning outcomes has been the subject of active interest in higher education for over 20 years but to date there is no approach which has led to a sustainable generalised solution to this problem.

A four staged measurement model is proposed which explores the learning outcomes specified by universities, establishes a set of standards against which such outcomes could be assessed, and examines local assessment of students’ learning for these outcomes to identify what graduates have learned and can do by the end of their study. Data on the grades achieved by individual students in local assessment tasks are then considered for use in a suite of institutional indicators which are designed to differentiate between universities in terms of the knowledge and skills demonstrated by their students. The focus of the study was to investigate whether the model could be applied to measure learning outcomes and institutional performance for Australian university undergraduate degrees.

The study showed that it was possible to derive a generalisable set of learning outcomes relevant to Australian universities and also a set of standards relating to each of these outcomes which could be used to grade assessments in a quantitative way for individual learning outcomes measurement. It was also possible to define a suite of quantitative performance indicators which appear to be valid for measuring differences in achievement for a subset of the specified learning outcomes. However it was discovered that Australian universities’ current practice in describing and testing learning outcomes for subjects rather than courses or for the institution is different to the approaches commonly used internationally, requiring an adjustment to the model. Universities’ practice in this is also different to the approach they espouse on their websites and in their assessment policies. The Australian approach requires a bottom-up model for measurement rather than the top-down model originally identified from international practice.

Various options are presented for types of local achievement assessment that are likely to produce the greatest consistency of learning outcome results between different universities. The favoured option is a set of newly devised signature assessments to test achievement of cognitive learning outcomes which could be framed in a discipline context, but this is a contentious solution. The bottom-up model has face validity based on detailed analysis of the expected outputs from each of its stages, but it could not be fully tested because assessment data held in universities’ repositories is not held at the level required.

Implementation of such a model, while appearing feasible, would have implications for policy, pedagogy, scholarship and practice within universities, and it would require a strong commitment from government and the sector for implementation to be successful. The benefits to students, staff, employers and the government would be substantial and appear to outweigh the costs associated with implementation.