Faculty of Education - Theses
Permanent URI for this collection
Search Results
1 - 1 of 1
-
ItemDelivery mode- does it matter?: comparing online to paper and pen delivery of a reading assessment( 2003)Collecting descriptive and diagnostic information about student progress is integral to informing teaching practice and supporting students in their learning. Many methods of assessment are available to teachers and ideally a range of appropriate assessment methods will be employed to gather such information. One popular method of assessment is a paper and pen assessment. Some paper and pen assessments are now also available in digital form and allow test delivery online. It is important for test users to know the answers to questions such as: is student performance comparable; are item difficulties comparable; and are there other factors influencing student performance in online mode? Research findings are somewhat divided about the importance of delivery mode and whether it is a significant factor in test difficulty. Findings vary from reporting little or no difference to reporting significant differences in student achievement across the modes. Some findings suggest that particular factors in the delivery mode may advantage or disadvantage the test taker. This study compared the difficulty of a reading assessment across two delivery modes - paper and pen and computer online delivery. It compared the difficulty of achieving like scores across the modes and also interrogated different item types to see how they behaved across the modes at the individual item level. Approximately 300 Year 7 students completed two reading assessments. All students completed the same initial paper and pen assessment. Approximately half the students then completed a second assessment in paper and pen mode, with the other half completing the same second assessment via online delivery. The first test, completed by all students, provided common items and thus the means to equate the paper and pen and online versions of the second test. The study found that it was not significantly harder or easier to achieve similar scores across the two modes and that, in general, item types behaved in similar ways across the modes. No significant differences were found related to item type or length of reading text. However three items were found to be significantly different in difficulty across the modes. For two of these items the difference can be attributed to formatting issues rather than mode specific reasons. Mode effect was the most likely explanation for the difference found in the difficulty of the other item Other items behaved differently across the modes and although the differences were not significant they did raise questions about online delivery that need to be further investigated.