Faculty of Education - Theses

Permanent URI for this collection

http://hdl.handle.net/11343/162

Filters

2007 1
1
Reset filters

Settings
Statistics
Citations
Author: Dewi, Handayani × Start date: 2007 × Subject: Evaluation ×

Search Results

Now showing 1 - 1 of 1
  • Item
    Thumbnail Image
    Using ability estimate trajectories to identify fatigue in the OECD PISA 2003 study
    Dewi, Handayani ( 2007)
    A test is designed to measure the performance of an examinee through the ability estimate obtained from the test. There are many factors that might challenge the validity of a test. Fatigue is one of these factors. Studies on test fatigue have been conducted since the 1930s. The awareness of this issue now seems to have declined, as it is difficult to find a recent paper focusing on fatigue. Currently the focus seems to be on `enhancing the reliability of the test', which also includes administering a longer test, thereby ignoring the silent threat of fatigue. This thesis studied the occurrence of fatigue in a test by examining ability estimate trajectories. Using ability estimates trajectories is a novel and innovative way of identifying fatigue in a test. An ability estimate trajectory is a plot of the means of the ability estimates of a group of testees from the fifth to the last item in the test. The data were obtained from the international-nonconsequential-test data of the Organisation for Economic Co-Operation and Development Programme for International Student Assessment (OECD PISA) 2003. This thesis analysed the ability estimate trajectories for the mathematics domain. The study found that the test was too long, and that fatigue existed in the test. Fatigue also occurred at different lengths of the test for different groups of the sample students (e.g., level of performance). It was considered that fatigue might also be related to student motivation and awareness of the stakes of the test - high stakes' tests providing stronger motivation thereby delaying the occurrence of fatigue. The findings, using the ability estimate trajectory, should assist test developers but indicate that caution should be exercised with test design, particularly for the challenge of low stakes' tests, e.g., for international studies and state curriculum monitoring assessment, as such tests have high visibility for decision makers.