University Library
  • Login
A gateway to Melbourne's research publications
Minerva Access is the University's Institutional Repository. It aims to collect, preserve, and showcase the intellectual output of staff and students of the University of Melbourne for a global audience.
View Item 
  • Minerva Access
  • Medicine, Dentistry & Health Sciences
  • Melbourne Medical School
  • Medical Education
  • Medical Education - Research Publications
  • View Item
  • Minerva Access
  • Medicine, Dentistry & Health Sciences
  • Melbourne Medical School
  • Medical Education
  • Medical Education - Research Publications
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

    Standard setting in Australian medical schools

    Thumbnail
    Download
    Published version (871.9Kb)

    Citations
    Altmetric
    Author
    Ward, H; Chiavaroli, N; Fraser, J; Mansfield, K; Starmer, D; Surmon, L; Veysey, M; O'Mara, D
    Date
    2018-04-23
    Source Title
    BMC Medical Education
    Publisher
    BIOMED CENTRAL LTD
    University of Melbourne Author/s
    Chiavaroli, Neville
    Affiliation
    Medical Education
    Metadata
    Show full item record
    Document Type
    Journal Article
    Citations
    Ward, H., Chiavaroli, N., Fraser, J., Mansfield, K., Starmer, D., Surmon, L., Veysey, M. & O'Mara, D. (2018). Standard setting in Australian medical schools. BMC MEDICAL EDUCATION, 18 (1), https://doi.org/10.1186/s12909-018-1190-6.
    Access Status
    Open Access
    URI
    http://hdl.handle.net/11343/254894
    DOI
    10.1186/s12909-018-1190-6
    Abstract
    BACKGROUND: Standard setting of assessment is critical in quality assurance of medical programs. The aims of this study were to identify and compare the impact of methods used to establish the passing standard by the 13 medical schools who participated in the 2014 Australian Medical Schools Assessment Collaboration (AMSAC). METHODS: A survey was conducted to identify the standard setting procedures used by participating schools. Schools standard setting data was collated for the 49 multiple choice items used for benchmarking by AMSAC in 2014. Analyses were conducted for nine schools by their method of standard setting and key characteristics of 28 panel members from four schools. RESULTS: Substantial differences were identified between AMSAC schools that participated in the study, in both the standard setting methods and how particular techniques were implemented. The correlation between the item standard settings data by school ranged from - 0.116 to 0.632. A trend was identified for panel members to underestimate the difficulty level of hard items and overestimate the difficulty level of easy items for all methods. The median derived cut-score standard across schools was 55% for the 49 benchmarking questions. Although, no significant differences were found according to panel member standard setting experience or clinicians versus scientists, panel members with a high curriculum engagement generally had significantly lower expectations of borderline candidates (p = 0.044). CONCLUSION: This study used a robust assessment framework to demonstrate that several standard setting techniques are used by Australian medical schools, which in some cases use different techniques for different stages of their program. The implementation of the most common method, the Modified Angoff standard setting approach was found to vary markedly. The method of standard setting used had an impact on the distribution of expected minimally competent student performance by item and overall, with the passing standard varying by up to 10%. This difference can be attributed to the method of standard setting because the ASMSAC items have been shown over time to have consistent performance levels reflecting similar cohort ability. There is a need for more consistency in the method of standard setting used by medical schools in Australia.

    Export Reference in RIS Format     

    Endnote

    • Click on "Export Reference in RIS Format" and choose "open with... Endnote".

    Refworks

    • Click on "Export Reference in RIS Format". Login to Refworks, go to References => Import References


    Collections
    • Minerva Elements Records [53102]
    • Medical Education - Research Publications [599]
    Minerva AccessDepositing Your Work (for University of Melbourne Staff and Students)NewsFAQs

    BrowseCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects
    My AccountLoginRegister
    StatisticsMost Popular ItemsStatistics by CountryMost Popular Authors