Faculty of Education - Theses
Permanent URI for this collection
Search Results
1 - 4 of 4
-
ItemAn investigation of Australian OECD Pisa trend results( 2009)This thesis investigates a range of equating-related issues for the Australian data collected under the Organisation for Economic Co-operation and Development (OECD) Programme for International Student Assessment (PISA). The implications for Australia's reported trend results are considered in detail. Following the exploration of differential item functioning (DIE) and dimensionality of the Australian PISA scales, a single scale, over all three PISA cycles (namely 2000, 2003 and 2006) for each major PISA domain (namely Reading, Mathematics and Science) was constructed. Previous published PISA results have employed a common reporting scale across all cycles for Reading, however scales common to all cycles have not been utilised for Mathematics or Science. Two further classes of equating issues are considered in this paper. First four different approaches to equating were used - two different treatments of missing data as well as two different item sets (all items and link items only) were estimated for each scale - and for each approach the implications for trends were discussed. Second, the equating approaches studied here used item parameters which are set at the country level rather than at the international level, thus allowing an examination of the impact of country DIF on the Australian trend results. Australian PISA trends were first explored in terms of means and standard deviations, and then in terms of the overall shape of the estimated performance distribution. This was achieved through the use of Q-Q (Quantile-Quantile) plots. Where applicable, comparisons were made with published trends. While results showed many similarities between models and published results, some differences were found. Australian PISA Reading means were statistically significantly lower when treating all omitted (or missing) responses as not administered at the item calibration stage compared to treating embedded omitted responses as incorrect and trailing omitted responses as not administered in PISA cycles 2003 and 2006. Between 2003 and 2006, published Australian Mathematics means were significantly lower than those found in this study. The published results showed a decline in means between 2003 and 2006, whereas the results reported here showed no change in the Australian means between these two cycles. Published Australian Reading distributions reported a decline from 2000 to 2003 and 2003 to 2006 in the number of Australian students located at the top end of the performance distribution. Between cycles 2000 and 2003 there was a decline from around the 70th percentile onwards and between cycles 2003 and 2006, the decline was even more severe; the higher the ability group the higher the decline from around the 20th percentile onwards. These estimated changes in the distribution shape were not replicated here, where the Australian data is analysed independently of the international data. The reanalysis undertaken here found a decline between the first two PISA cycles, but remarkably in the bottom 15 per cent of the distribution only. Between cycles 2003 and 2006 an almost constant decline across the whole proficiency distribution was found and not a decline that was limited to the top end of the distribution. The reported results highlight some of the potentially important differences that can occur when different analysis methods are used.
-
ItemImplementation and implications of the National English Profile for improving assessment strategies in a primary classroom( 1996)The National English Profile provides the framework for teachers to use in assessing the literacy development of children in Australian schools. This study documents how one school and, in particular, one teacher, implemented the National English Profile. In order to place this initiative in the context of educational change in Victoria today, recent initiatives and changes to the teaching of language have been examined. Current literature in the field of literacy and assessment have been critically reviewed. As a foreground to the study, the process of educational change is examined in two interrelated case studies. The first looks at how the site school reviewed their Assessment and Reporting Policy. The second examines personal educational change of the researcher with regard to the teaching and assessment of literacy. Both of these case studies refer to current literature in the field and refined the focus of the research in one classroom. This research study draws on and investigates a national initiative in assessment. It traces the implementation and implications of the National English Profile on classroom based assessment strategies. It took place over two years, at a time of quite dramatic change in state education in Victoria. The study provides insight into how teachers managed and adjusted to these changes in one particular school. The research design is drawn from naturalistic inquiry methodology. Data were collected from teachers, parents and children of the site school through questionnaires and interviews. From an analysis of these data, generalisations were made about the understandings a group of teachers, parents and children have about the worth of the National English Profile and the usefulness of particular classroom based assessment strategies. The use of Portfolios as a system of work sample collection that is both manageable and purposeful for teachers and children has been examined in this study.
-
ItemFrom "great!" to "gr8!" : register shifts across CMC modes : a case study of three boys with varying literacy abilities( 2007)The aims of the current study were to observe A) the extent to which various ability level students demonstrate control over register in the three CMC modes of SMS, email and Word documents; and B) the extent to which these students are aware of these shifts in register. Data was collected by using a video recorder to capture how three students, Alex, Jasper and Eric, typed an SMS, an email, and a school-based text as a Microsoft Word document. Each student then discussed their texts in an interview, using the footage as a think-aloud protocol. All sessions were carried out with individual students, each lasting no more than 20 minutes. Data were analysed using the Hallidayan theory of register, taking into account its sub-categories of field, tenor and mode (Halliday, 1985a; 1985b; Eggins, 2004). The video footage indicated that Alex and Jasper, the students with higher literacy abilities, were able to switch between abbreviated codes and academic written English, demonstrating an understanding of the different communicative requirements of both informal and formal texts. The linguistic analysis illustrated that these students were very comfortable in using both Shorthand codes and conventional spelling in their SMS and email messages, and demonstrated a sophisticated use of the generic, syntactic and especially, lexical structures in their formal texts. However, Eric, identified by his teacher as having lower literacy abilities, was less comfortable in using abbreviations in his SMS and email; a less sophisticated structure in his formal text even showed some register confusion. The interviews revealed Alex's and Jasper's awareness of their register choices, evident across their texts. Although Eric also showed an awareness of these, his texts show he demonstrated these shifts to a lesser extent. Alex and Jasper also revealed positive attitudes towards, and an exploratory use of CMC modes. However, Eric's surprising aversion to technology suggests that some teenagers may be uncomfortable with the abbreviated language associated with some forms of CMC. Results show that, contrary to populist beliefs, (such as Texting 'is not bar to literacy;' cited in Thurlow, 2006, p. 699) the relationship between Shorthand codes and academic written English is multifaceted. Alex's and Jasper's awareness of composing texts with varied tenor allowed them to make appropriate register choices, whereas Eric's choices of code were more restricted. This study suggests that clearly, there are more complex variables operating, which should be examined in a wider array of students in future investigations.
-
ItemAssessing critical reading performance in an EAP classroom( 2003)A widespread perception among EAP teachers and course designers in Australia is that Asian university students need some formal training in text analysis and critical thought. To be more precise, Asian students are often portrayed as victims of Asian education, where students are taught to rote learn and memorise, rather than question and analyse as they might be taught in Australia. EAP classes often rest on generalisations about cultural differences, constructing critical reading tasks on the assumption that the problems students encounter in completing critical reading tasks relates to the way they read rather than how they respond to what they read (that is, their familiarity and ability to reproduce arguments in acceptable and culturally expected ways). While EAP researchers have spent considerable energy on improving critical reading methodologies, there has been very little research on the assessment of critical reading in EAP classes. In particular, very little research has offered any explanation of what might constitute evidence of learning to read and respond at a high level. In this thesis, I develop a criterion-referenced critical reading performance rubric, informed by Ballard and Clanchy's (1984; 1988; 1997) research on critical analysis and Toulmin's (1958; 1972; 1984) theories on reasoning and argumentation, to examine the clarity, evidence and logic (CEL) of students' critical reading responses and the influence of text selection on student reasoning skills. Secondly, I explore with students some factors they perceive as affecting their reading performances including text selection. The findings of this research reveal that there are distinct patterns in critical reading performances characterising low, medium and high level students, and that subject matter may be less significant to students' reading performances than they seem to believe. In discussing critical reading performances with students, they identified several factors such as a lack of subject knowledge, a lack of time and a misunderstanding of their purpose in completing the task which significantly affected their performance. Additionally, students perceived their future performances could be improved by choosing texts with subject matter more closely aligned to their academic field. The findings also reveal that most students feel that critically examining topics relating to Australian social issues are not helpful to the overall development of their critical reading skills.