Faculty of Education - Theses
Permanent URI for this collection
Search Results
1 - 5 of 5
-
ItemApplications of item response theory to identify and correct for suspect rater data( 2006)This thesis describes a plausible values imputation approach for deriving population estimates on several language proficiency domains. The approach harnessed a multi-dimensional item response analysis combining student responses, rater judgements and student background variables. The target student population was lower grade primary school students enrolled in the Hong Kong schooling system. The raters consisted of local teachers of English employed within the sampled target schools. The primary objective of this research was to impute plausible values where no data was provided or where rater data was deemed suspect. By necessity, a secondary objective of this study was to establish rules for justly excluding particular data on the basis of questionable validity. Surveys such as TIMSS, PISA and NAEP have used such "plausible value" methodologies to account for incomplete test designs and person non-response (Beaton & Johnson, 1990; Yamamoto & Kulick, 2000; Adams & Wu, 2002). The point of difference between this study and other similar studies was the use of item response theory (in particular plausible values imputation) to identify and correct for invalid rater judgements in a large-scale educational survey. An additional research outcome included a derived index of rater data quality based upon imputation scores.
-
ItemSearching for a fair assessment( 1995)The full implementation of the VCE in 1992 represented significant changes to senior secondary schooling in Victoria. These changes were strikingly evident in the new assessment methods and procedures of the Art test CAT. The difference between the HSC Art Appreciation examination process and the 1992 VCE Art test CAT assessment was more than a difference in practices and methods. The ideological foundations of the two examination systems were vastly different. The HSC system served to rank students so that universities could select the best candidates. The BOS promoted the VCE as more 'equitable', 'accountable' and 'inclusive 'than the HSC. These concepts derived from the recommendations of the Blackburn Report (1985) which advocated major reform of the senior curricula and assessment. I interviewed five experienced Art Appreciation examiners in order to acquire a perspective of the two examination systems during' a time of change (1987-92) and ultimately focused upon the fairness values that were evident in the examination process. I compared published policy statements regarding fairness and equity with the examination practices that the five examiners described and analysed during our interviews. The theories of Foucault and Bourdieu enabled me to understand that examinations are part of a systematic social selection process. How can 'fairness' be defined and embodied in examination practices if examinations inherently represent a form of social power? Bourdieu and Foucault nourished this type of reflection. The Blackburn Report, the views of the five examiners and the social theories of Foucault and Bourdieu led me to consider the Art test CAT (1992) as a representation of the equity values that were defended by the BOS. Fullan's theories revealed that disorder usually accompanies educational change and added -another dimension to this investigation of 'fairness and justice' in the Art Appreciation examination (1987-92). In addition, this research study evaluates the practice of the BOS policy regarding equity and fairness, as evident in the VCE Art test CAT assessment process (1992), because the Board promoted the view that the VCE would provide greater equity and fairness to all Victorian students than the HSC.
-
ItemCaught in the middle : teachers' perceptions of the usefulness of the Record of oral language when used with ESL students( 2004)In the present educational context, large-scale literacy reform has been hotly debated in the media by various stakeholders while on the other hand the " voice that has been largely absent from these debates has been that of teachers". This study attempts to correct this imbalance by reporting on a study of teachers' perceptions of the usefulness of the ROL as a testing tool for students learning English as a second language. It was designed to offer the researcher, the participants and other stakeholders great insight into the current complexities that teachers face when assessing the oral competence of students learning English as a second language. Teachers were given the opportunity to critically examine their own assumptions, beliefs and practices about oral language learning and assessment and challenge the current use of the ROL in the Children's Literacy Success Strategy (CLaSS) program. A qualitative case study approach was selected to allow the voices of the participants and the researcher to be heard in a familiar setting. Thirty minute semi-structured interviews were conducted separately so that teachers could speak freely thus enabling the researcher to follow individual trains of thought and obtain in-depth responses. The study concluded that the usefulness of the Record of Oral Language as a testing tool for students from language backgrounds other than English was not the central issue. Research findings instead highlighted the difficulties faced by mainstream teachers expected to effectively assess the oral language skills of students for whom English is a second language without effective Professional Development support. The central issue that has emerged in this study is the place of teacher professional knowledge for these teachers of ESL students. Not only are ESL students disadvantaged in the current educational climate but so are the mainstream teachers. Teachers' responses to this study reaffirm the need for Professional Development in ESL pedagogy as a way to effectively support the Oral Language and Literacy Learning of ESL which was valued pre Literacy Advance.
-
ItemAnalysing and treating test item nonresponse( 2001)This study uses an empirical and normative methodology to analyse the effects and various treatments of nonresponse on the measurement of student ability in large scale educational achievement studies. The primary focus of the study is an empirical analysis of disturbances introduced into psychometric measurement by the presence and treatment of nonresponse. This focus is balanced by a normative analysis of the possible implications of a series of psychometric treatments of nonresponse on students' response strategy and on the conduct of standardised testing. Initially, the scope, focus and research questions of the current study are outlined. Second, the research perspectives used to study nonresponse and used in this thesis are reviewed. Third, nonresponse is defined and distinguished from other forms of test item missing data. Methods proposed for analysing and treating test item nonresponse are considered, and the six treatments examined in this thesis are presented. The methodology underpinning the current analysis is then presented. This study investigates the empirical performance of each of these treatments using data from a crossnational test of Reading ability. After presenting the characteristics of the test and sample, the assumptions and methods underpinning analysis of the treatments are described. First, item response theory and Rasch measurement are discussed. Second, the nonresponse treatments are considered in more depth. Finally, the psychometric and statistical methods used to evaluate the treatments are presented. Results describing the distributions of nonresponse in the samples and the characteristics of student ability estimates under each of the treatments are presented. Next, the psychometric, response strategy and situational implications of each treatment are discussed. The relative merits of these treatments are summarised in conclusion, and implications for research and practice are considered.
-
ItemThe influence of cognitive style on student performance and preferences in mathematical assessment( 2003)This study examined the influence of cognitive style on student performance and preferences in mathematics assessment. Two dimensions of cognitive style are: the wholistic-analytic style dimension and the verbal-imagery style dimension. This study investigated the comparative influence of these dimensions, students' _display of mathematics knowledge, specifically, a fractions test, in different assessment and their preferences toward mathematical assessment. A sample of 74 year seven students aged eleven to fourteen (n = 49 males and n = 25 females) with a mean age of twelve years and one month completed an assessment of their knowledge of fractions. Test items were presented verbally, symbolically and in pictorial formats. As well, students' preferences to aspects of mathematics learning and assessment were examined using a questionnaire. A Cognitive Styles Analysis (CSA) instrument was used to assess students' cognitive styles. These protocols were formed as the basis for the research. Cognitive style was found, to some extent, to influence student preferences in mathematical assessment more than student performance in mathematics. There was a significant partial correlation between a cognitive style and student preferences in mathematics assessment; analytic learners preferred the common test and wholistic learners preferred the project. Specifically, (i) analytic learners preferred the common test to the project, as a way of seeing themselves making progress and as a way of thinking about learning mathematics; and (ii) wholistic learners preferred the project to the common test, as a way of identifying the degree they feel pressured and as a way of thinking about learning mathematics. There was indefinite correlation between cognitive style and student performance in mathematics; analytic learners did not perform better than wholistic learners in the fractions test. Further, analytic learners did not perform better on pictorial tasks than on symbolic tasks. Similarly, wholistic learners did not perform better on symbolic tasks than on pictorial tasks. The implication of these findings for teaching is that cognitive style does not influence students' performance in mathematics. This study allowed the author to glimpse at the influence of cognitive style on students' preferred ways of being assessed, through the windows of ways of learning, mathematics knowledge and mathematics assessment.