Melbourne Graduate School of Education - Theses

Permanent URI for this collection

Search Results

Now showing 1 - 1 of 1
  • Item
    Thumbnail Image
    Assesing the spoken skills of Japanese secondary students
    Akiyama, Tomoyasu ( 1999)
    This thesis has evaluated a test of spoken English for secondary school students in Japan in relation to its validity, reliability and practicality. Although the Japanese Ministry of Education has put a strong emphasis on the improvement of communicative competence, little research and development work has been carried out on speaking tests in particular those tests administered to secondary school students in Japan. Recent research on Item Response Theory in language testing has primarily investigated test validity and reliability for adults and immigrants in a second language context (e.g. Brown 1995, McNamara 1990 a, b, O'Loughlin 1997). The present study has applied Item Response Theory to analyses of a speaking test where English is learned as a foreign language in the classroom. The various types of data used in this study were gathered from a test administered to 109 students in Tokyo in December 1998. The test was conducted by five interviewers, and seven raters rated each student's performance independently. Analyses of candidate ability, item difficulty and rater severity were carried out with the Item Response Modelling Program 'ConQuest' (Wu, Adams and Wilson 1998). Questionnaire responses from test takers and interviews were also analysed. Overall, the results showed that the test possessed high validity, reliability and practicality in the context for which it was developed. In particular, item analysis and feedback were found of crucial value in providing evidence of validity. It was also found that there was variability in the relationship between individual items and raters in terms of rater severity. These findings have demonstrated the need to conduct further research on how task difficulty affects student performance and the relationship between the washback effect and student speaking ability in Japan.