Faculty of Education

Search Results

Now showing 1 - 1 of 1

Applications of item response theory to identify and correct for suspect rater data

Zoanetti, Nathan Paul ( 2006)

This thesis describes a plausible values imputation approach for deriving population estimates on several language proficiency domains. The approach harnessed a multi-dimensional item response analysis combining student responses, rater judgements and student background variables. The target student population was lower grade primary school students enrolled in the Hong Kong schooling system. The raters consisted of local teachers of English employed within the sampled target schools. The primary objective of this research was to impute plausible values where no data was provided or where rater data was deemed suspect. By necessity, a secondary objective of this study was to establish rules for justly excluding particular data on the basis of questionable validity. Surveys such as TIMSS, PISA and NAEP have used such "plausible value" methodologies to account for incomplete test designs and person non-response (Beaton & Johnson, 1990; Yamamoto & Kulick, 2000; Adams & Wu, 2002). The point of difference between this study and other similar studies was the use of item response theory (in particular plausible values imputation) to identify and correct for invalid rater judgements in a large-scale educational survey. An additional research outcome included a derived index of rater data quality based upon imputation scores.