Assessor grades and comments: private thoughts and public judgements

Abstract

Assessment of medical trainees’ performance in the workplace aims to provide them with accurate and meaningful information and guidance on their learning and developing competence. However, in practice, such goals aren’t always achieved. Sometimes assessors find it difficult to deliver clear and consistent assessment messages to a trainee, especially when the information or judgement they have to give is negative. While this can occur for many reasons, including disagreement with or uncertainty about assessment processes, the MUM effect — the widespread human tendency to keep mum about undesirable messages — may also have relevance for the situation. With reference to this framework, this thesis explores how and why reluctance to deliver negative assessment messages manifests in a medical specialty training setting in Australia.

Literature reviews on the MUM effect and trainee perspectives of assessment messages informed the design of a mixed methods study which explores the MUM effect in this context. The study involved four parts: - a questionnaire study of assessor self-reports of discomfort and MUM behaviours in assessment; - a questionnaire study of trainee perspectives of MUM behaviours by their assessors and their views of the clinical performance assessments; - a review of a sample of previously submitted assessment forms comparing the messages sent by numerical ratings with those by written comments; and - an interview study of assessors to further understandings of their experiences with and perspectives of these assessment formats.

The findings show that reluctance to deliver negative assessment messages — which can result in failure to give feedback, failure to fail and grade inflation — are real and continuing issues in medical education. The MUM effect offers one explanation for their persistence, despite the many methods which have been employed to date to address them. The study shows how the MUM effect permits an expanded view of the problem and that assessor reluctance can lead to behaviours beyond the commonly reported failure to fail and grade inflation. These include behaviours such as delay, avoidance and distortion of assessment information. Further, the results show that reluctance can affect the comments part of an assessment in addition to the ratings, which have been the main focus to date. This study reveals the many pressures and dilemmas that assessors face in their role and in particular, that the amount of discomfort they experience can potentially affect their assessment behaviours and result in MUMing. This work also shows that trainees are aware that their assessors sometimes keep mum, meaning the judgement delivered may differ from the assessor’s private thoughts on their performance. Potential solutions are seen to be multifactorial and include addressing perceptions about “failure” in clinical performance assessments and the responsibility that assessors feel for the assessment decisions.