Melbourne Graduate School of Education - Theses

Permanent URI for this collection

Search Results

Now showing 1 - 1 of 1
  • Item
    Thumbnail Image
    Laying the foundations for surgical excellence: strategic alignment of questions in program evaluation
    Martin, Jenepher Ann ( 2006)
    The question of how to prioritise program evaluation questions in the best way is an important issue for evaluation practice. This is because the prioritisation of evaluation questions can determine how resources are allocated to the program evaluation effort. Obviously, if critical questions are not addressed, then the information obtained may not be the most useful in terms of program improvement. In the area of Program Theory Evaluation (PTE), prioritisation of questions, emerging from a program theory, has been highlighted as a need. This thesis used case study methodology to investigate a novel method of prioritising evaluation questions in a program theory evaluation, with the intention of creating a strategically-aligned evaluation plan. The case study concerned the Basic Surgical Training (BST) program of the Royal Australasian College of Surgeon (RACS). The thesis set out first to develop the program theory for the BST program, and to construct a balanced score card (BSC) with respect to education within the RACS. Subsequently, the aim was to develop a strategically aligned evaluation plan for BST using the BSC as a quality framework of reference. Multiple data collection methods were used including document analysis, focus group interviews, and individual interviews. The findings from the data analysis were used as a basis for both the BST program theory and the BSC, termed the "RACS Education Quality Framework". The application of the BSC to the program theory then resulted in an evaluation plan for an aspect of the BST program, which was considered to be a priority by the RACS. The outcomes of this thesis illustrate the utility of this approach in prioritising evaluation questions in program theory evaluation. There are, however, a number of caveats that relate primarily to the commitment needed to develop the BSC and the complex program theory. Despite this, a number of recommendations relating to evaluation within the RACS, evaluation practice in medical education settings, and evaluation practice more widely have been made. Finally, the processes and outcomes described in the thesis provide the basis for further exploration of prioritisation of questions in program theory evaluation, and the concepts explored should have wider applicability to evaluation practice. Further work in other case settings would assist in defining the utility of this approach.