Faculty of Education - Theses
Permanent URI for this collection
Search Results
1 - 1 of 1
-
ItemNumeracy assessment: from functional to critical practice( 2004)This study examines the validity of the measures obtained from a performance assessment of students' developing context in a numeracy context. The study was based on three premises: that teachers could make valid and reliable judgments about their students in regular classroom situations; that numeracy competence was developmental and involved higher order thinking; and that using different performance tasks could provide information about changes in students' performances over time. Numeracy was ill-defined and no developmental progression in numeracy was available on which to base assessment tasks. Common elements in descriptions of numerate behaviour and related ideas, suggested that numeracy could be conceptualised as a competence, and addressed through a generalised developmental progression. A competency perspective implied that teachers would be actively involved, and that assessment and teaching would be the same process. Performance assessment met some of these conditions, but the participation of teachers in performance assessment implemented in unstandardised conditions in regular classrooms raised validity issues. In this study, teachers used clearly specified scoring rubrics that emphasised the quality of students' performances to assist their observations of their students' behaviour when undertaking assessment tasks in numeracy contexts. The teachers' assessment took a developmental perspective that included higher order thinking, such as generalisation, justification and conjecture. Performance assessment tasks were designed using a set of design rules developed after initial trialling of pilot tasks. The different performance assessment tasks were underpinned by a generalised continuum of competence and set in diverse numeracy contexts. One cohort of 1243 Year 10 students undertook a single performance assessment task, and tests of mathematics skills, mathematical problem solving, and English ability, to provide convergent and discriminant evidence of construct validity. These students were in classes taught by 32 teachers in 14 different schools. Groups of students in Years 8 to 10 also undertook five different performance assessment tasks to provide additional validity evidence and longitudinal data. The total number of students' responses involved was 3412. All assessments were scaled using Rasch measurement techniques. Consideration of fit to the Rasch model indicated that all activities on the performance tasks, based on teachers' judgments of their students' performances, worked together in a predictable fashion to address a unidimensional underlying construct. Interpretation of this variable indicated that the demands of the activities closely matched the levels or the generalised continuum of competence. Convergent and discriminant evidence from a Multitrait Multimethod matrix, confirmed by Structural Equation Modelling approaches, indicated that there was no undue method effect from the use of teacher-judged tasks. Further findings showed that the teacher-judged performance task produced consistent information across all classes, within all schools involved. This was interpretable from a single perspective provided by the underlying generalised continuum of competence. This continuum addressed a wide range of ability levels, and included higher order thinking at the upper levels. Changes in students' performances over time could be monitored by reference to levels of development within the continuum of competence. The assessment approach, based on tasks planned specifically to link the task activities to an underlying continuum of competence through the use of carefully designed rubrics, allowed higher order thinking in numeracy contexts to be demonstrated by the students, and provided evidence of changes in students' performances over time. The findings suggested that assessment practice based on a competence approach to developmental assessment in numeracy contexts could provide valid outcome measures when implemented by teachers within regular classrooms. In addition, the process has the potential to provide teachers with information about the point at which teaching intervention could be maximally effective.