Melbourne Graduate School of Education - Research Publications

Permanent URI for this collection

Search Results

Now showing 1 - 10 of 20
  • Item
    Thumbnail Image
    Actual Interpretations and Use of Scores as Aspects of Validity
    O'Leary, TM ; Hattie, JAC ; Griffin, P (WILEY, 2017-06-01)
    Validity is the most fundamental consideration in test development. Understandably, much time, effort, and money is spent in its pursuit. Central to the modern conception of validity are the interpretations made, and uses planned, on the basis of test scores. There is, unfortunately, however, evidence that test users have difficulty understanding scores as intended. That is, although the proposed interpretations and use of test scores might be theoretically valid they might never come to be because the meaning of the message is lost in translation. This necessitates pause. It is almost absurd to think that the intended interpretations and uses of test scores might fail because there is a lack of alignment with the actual interpretations made and uses enacted by the audience. Despite this, there has only recently been contributions to the literature regarding the interpretability of score reports, the mechanisms by which scores are communicated to their audience, and their relevance to validity. These contributions have focused upon linking, through evidence, the intended interpretation and use with the actual interpretations being made and actions being planned by score users. This article reviews the current conception of validity, validation, and validity evidence with the goal of positioning the emerging notion of validity of usage within the current paradigm.
  • Item
    Thumbnail Image
    Fixed or mixed: a comparison of three, four and mixed-option multiple-choice tests in a Fetal Surveillance Education Program
    Zoanetti, N ; Beaves, M ; Griffin, P ; Wallace, EM (BMC, 2013-03-04)
    BACKGROUND: Despite the widespread use of multiple-choice assessments in medical education assessment, current practice and published advice concerning the number of response options remains equivocal. This article describes an empirical study contrasting the quality of three 60 item multiple-choice test forms within the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG) Fetal Surveillance Education Program (FSEP). The three forms are described below. METHODS: The first form featured four response options per item. The second form featured three response options, having removed the least functioning option from each item in the four-option counterpart. The third test form was constructed by retaining the best performing version of each item from the first two test forms. It contained both three and four option items. RESULTS: Psychometric and educational factors were taken into account in formulating an approach to test construction for the FSEP. The four-option test performed better than the three-option test overall, but some items were improved by the removal of options. The mixed-option test demonstrated better measurement properties than the fixed-option tests, and has become the preferred test format in the FSEP program. The criteria used were reliability, errors of measurement and fit to the item response model. CONCLUSIONS: The position taken is that decisions about the number of response options be made at the item level, with plausible options being added to complete each item on both psychometric and educational grounds rather than complying with a uniform policy. The point is to construct the better performing item in providing the best psychometric and educational information.
  • Item
    No Preview Available
    Measuring Collaborative Problem Solving Using Mathematics-Based Tasks
    Harding, S-ME ; Griffin, P ; Awwal, N ; Alom, M ; Scoular, C (American Educational Research Association, 2017-07)
    This study describes an online method of measuring individual students’ collaborative problem-solving abilities using four interactive mathematics-based tasks, with students working in pairs. Process stream data were captured from 3,000 students who completed the tasks in the United States, Australia, Canada, Costa Rica, Singapore, and Finland. The data were transformed into indicators of collaborative problem-solving ability and were analyzed using item response modeling. The assessments employed in this study can be used as a teaching tool for introduction to algebraic concepts and as a measurement instrument for collaborative problem-solving ability. The paper describes the construction, calibration, and reliability of the tasks and considers validation issues, such as fairness between assessments for both partners and avoidance of cultural biases. Investigations into the dependencies between student scores provide evidence for convergent and discriminant validity.
  • Item
    Thumbnail Image
    Formative Assessment Project - Phase 1 Research Report
    Cagasan, L ; Griffin, P ; Nava, FJ ; Vista, A ; CARE, E (Assessment, Curriculum and Technology Research Centre (ACTRC), 2016)
  • Item
    Thumbnail Image
    Profiling developmental pathways of communication and literacy for students with additional learning needs
    WOODS, K ; GRIFFIN, P (Australian Association of Special Education Inc, 2008)
  • Item
    Thumbnail Image
    What are the main cross-national studies?
    Grisay, A ; GRIFFIN, P ; Ross, K ; Genevois, I (UNESCO, 2006)
  • Item
    Thumbnail Image
    An integrated multi-source measurement approach to the assessment of management competencies
    GRIFFIN, PE ; GILLIS, S ; REDDY, S (ICFAI University Press, 2005)
  • Item
    Thumbnail Image
    Teachers' Use of Assessment Data
    Griffin, P ; WyattSmith, C ; Cumming, JJ (SPRINGER, 2009)
  • Item
    Thumbnail Image
    Competency Assessment
    GILLIS, S ; GRIFFIN, P ; Athanasou, J (David Barlow Publishing, 2008)
  • Item
    Thumbnail Image
    General characteristics of Grade 4 teachers and their teaching
    GRIFFIN, P ; Postlethwaite, T ; Zhang, Y ; Postlethwaite, T ; Grisay, A (UNESCO Institute for Statistics, 2008)