School of Languages and Linguistics - Theses
Permanent URI for this collection
Search Results
1 - 2 of 2
-
ItemThe importance and effectiveness of moderation training on the reliability of teacher assessments of ESL writing samples( 1993)This thesis reports the findings of a study of the inter-rater reliability of assessment of ESL Writing by teachers in the Australian Adult Migrant Education Program, using the ASLPR, a language proficiency scale used throughout the program. The study investigates the individual ratings assigned to 15 writing samples by 83 teachers, both before and after training aimed at moderation of raters' perceptions of descriptors in the scale by reference to features of other 'anchor' writing samples. The thesis argues the necessity for on-going training of assessors of ESL writing, at a time of change in the program, from assessment of language proficiency to that of language competencies, since both forms of assessments are increasingly having consequences which affect the lives of the candidates. The importance and necessity for moderation training is established by reference to the problems of validity in the scale itself and in its use in the program, and by reference to the literature of assessor-training and features of writing which influence rater-judgements. The findings indicate that training is effective in substantially increasing inter-rater reliability of the subjects, by reducing the range of levels assigned to the samples and increasing the percentages of ratings at the mode (most accurate) level and at the Mode +/- 1 level (an allowance for 'error' due to the subjective nature of the assessment), after training. The paper concludes that on-going training is effective in achieving greater consensus i.e. inter-rater reliability amongst the assessors, but suggests that variability needs to be further reduced and offers suggestions for further research aimed at other assessors and variables.
-
ItemThe predictive validity of the IELTS and TOEFL: a comparison( 1994)This study compared two groups of overseas students who entered Melbourne and Monash universities in Melbourne in semester 1 1993. One group entered on the basis of an IELTS score and the other group entered on the basis of TOEFL score. Their academic performance at the end of semester 1 1993 was compared. Predictive validity coefficients were also compared. Differences were minimal with a slight tendency for the TOEFL to correlate more strongly than IELTS with undergraduate academic performance. The assumption made by admissions officers who use the two tests to make admissions decisions that the two tests are equivalent in their predictive validity was not refuted.