School of Languages and Linguistics - Research Publications
Permanent URI for this collection
Search Results
1 - 10 of 24
-
ItemA Guide to English language policy making in higher education(International Education Association of Australia (IEAA), 2021-11-02)In this webinar, we will be presenting a series of 11 guides designed for Admissions and Compliance teams to understand the key elements of language learning in relation to higher education. The information in the guides can be used to make informed judgements about the types of English language tests that might be best suited for admitting students into programs.
-
ItemThe challenges of providing expert advice in policy contexts(Association for Language Testing and Assessment of Australia and New Zealand (ALTAANZ-University of Melbourne), 2021-01-01)Language testers can have various roles in relation to the use of language tests and policy. One such role may be to provide expert advice in policy formation or policy review contexts. Such instances are often not documented systematically, as advice may be provided in informal or closed meetings, or confidential documents, which are not available to the public. The challenges associated with providing such advice are also rarely recorded. In this paper, I describe three instances in which the Language Testing Research Centre (LTRC) at the University of Melbourne was invited to provide external policy advice. I specifically reflect on what prompted the invitations, what advice we were asked to provide, the complexity of providing advice in each instance, and whether the advice was taken up by the policy makers. The paper reflects on some of the common threads and concludes with implications for training new language testers.
-
ItemTrust the test: Score-user perspectives on the roles of language tests in professional registration and skilled migration(Association for Language Testing and Assessment of Australia and New Zealand (ALTAANZ-University of Melbourne), 2021-01-01)Englishlanguage proficiency is a deciding factor in the life opportunities of many thousands of applicants for Australian skilled migration every year. This paper focuses on the perspectives of professional bodies that use English language tests in their decisions. Taking an interpretative approach, we explore the meanings that policy makers from these organisations ascribe (as score users) to test standards (cut-scores) so that we can better understand the uses of test scores in migration policy. The policy narratives we observed around the use of test scores describe the need to manage large numbers of applicants, to assure a level of English proficiency for high-risk professional communications, to provide an objective assessment that is separate from any assessment of professional competence and to maintain consistency of standards with other bodies. These views are contextualised with other relevant information, particularly that available from test providers, who are key players in the test-using interpretive community. We observe that particular tests and their standards become trusted and entrenched in policy, using the apparently simple semiotics of scores. Concomitantly, trust in tests is nurtured by test marketing. These tendencies warrant attention from test researchers, providers and score users.
-
ItemNo Preview AvailableRevisiting rating scale development for rater-mediated language performance assessments: Modelling construct and contextual choices made by scale developers(SAGE PUBLICATIONS LTD, 2021-10)Rating scale development in the field of language assessment is often considered in dichotomous ways: It is assumed to be guided either by expert intuition or by drawing on performance data. Even though quite a few authors have argued that rating scale development is rarely so easily classifiable, this dyadic view has dominated language testing research for over a decade. In this paper we refine the dominant model of rating scale development by drawing on a corpus of 36 studies identified in a systematic review. We present a model showing the different sources of scale construct in the corpus. In the discussion, we argue that rating scale designers, just like test developers more broadly, need to start by determining the purpose of the test, the relevant policies that guide test development and score use, and the intended score use when considering the design choices available to them. These include considering the impact of such sources on the generalizability of the scores, the precision of the post-test predictions that can be made about test takers’ future performances and scoring reliability. The most important contributions of the model are that it gives rating scale developers a framework to consider prior to starting scale development and validation activities.
-
ItemThe Effect of Sustained Teacher Feedback on CAF, Content and Organization in EFL Writing(Urmia University, 2020-06-01)Despite teachers' mainstream practices in L2 writing classrooms addressing different dimensions of writing over time, much of the research on feedback in recent years has been of relative short duration and has mainly focused on accuracy. The current longitudinal study investigated the influence of sustained teacher written feedback on accuracy, syntactic complexity, fluency, content, and organization in an EFL context. Ninety-two learners were divided into four groups, receiving written corrective feedback, feedback on content and organization, multilateral feedback (i.e., on grammatical accuracy, content, and organization), and no feedback over a 3½-month period. They completed a pre-test, post-test, and delayed post-test and wrote and revised eight interim expository compositions on a weekly basis. Results showed that the three treatment groups significantly improved in the dimensions on which they received feedback. However, only the groups who received feedback on content and organization improved in fluency. Importantly, the multilateral group improved in accuracy as well as fluency, content and organization. Theoretically, the findings endorse the language learning potentials of sustained writing as long as it is guided by teacher feedback. The findings provide empirical support for the influence of sustained feedback on expanding and consolidating learners’ explicit knowledge of L2 writing.
-
ItemThe impact of national standardized literacy and numeracy testing on children and teaching staff in remote Australian Indigenous communities(Sage Publications, 2019)All educational testing is intended to have consequences, which are assumed to be beneficial, but tests may also have unintended, negative consequences (Messick, 1989). The issue is particularly important in the case of large-scale standardized tests, such as Australia’s National Assessment Program - Literacy and Numeracy (NAPLAN), the intended benefits of which are increased accountability and improved educational outcomes. The NAPLAN purpose is comparable to that of other state and national ‘core skills’ testing programs, which evaluate cross-sections of populations in order to compare results between population sub-groupings. Such comparisons underpin ‘accountability’ in the era of population-level testing. This study investigates the impact of NAPLAN testing on one population grouping that is prominent in the NAPLAN results’ comparisons and public reporting: children in remote Indigenous communities. A series of interviews with principals and teachers documents informants’ first-hand experiences of the use and effects of NAPLAN in schools. In the views of most participants, the language and content of the test instruments, the nature of the test engagement, and the test washback have negative impacts on students and staff, with little benefit in terms of the usefulness of the test data. The primary issue is the fact that meaningful participation in the tests depends critically on proficiency in Standard Australian English (SAE) as a first language. This study contributes to the broader discussion of how reform-targeted standardized testing for national populations affects sub-groups who are not treated equitably by the test instrument or reporting for accountability purposes. It highlights a conflict between consequential validity and the notion of accountability that drives reform-targeted testing.
-
ItemCulturally and linguistically diverse populations in medical research: perceptions and experiences of older Italians, their families, ethics administrators and researchers(SAGE PUBLICATIONS INC, 2016)BACKGROUND: Low-participation of culturally and linguistically diverse (CALD) patients in medical research remains a problem in migrant and refugee destination countries such as Australia. The aims of this study were to explore i) CALD persons' perceptions and experiences of the medical system and medical research, in this case, older Italian Australians; and ii) the views of research professionals on CALD patient participation in medical research. DESIGN AND METHODS: A qualitative study was conducted in Melbourne, Australia, in 2015 utilising in-depth interviews and focus groups with four stakeholder groups: older Italian Australians (n=21); adult children of older Italian Australians (n=10); hospital Human Research Ethics Committee administrators (n=4); and clinical researchers (n=4). The data were analysed for content and thematic analysis. RESULTS: Themes for the CALD and family group were getting by in medical interactions; receptivity to medical research: testing the waters; and, receptivity to technology for support: passive versus active. Themes for the researcher and HREC groups about CALD patient participation in research were: exclusion; cultural factors; and e-consent. CONCLUSIONS: Our findings from four stakeholder perspectives and experiences confirm that there were considerable cultural, linguistic, and resourcing barriers hindering the participation of older Italian-Australians in medical research. Furthermore, our findings showed that in this study setting there were few enabling strategies in place to address these barriers despite the national ethics guidelines for equitable participation in research. The findings informed the creation of a multimedia tool whose purpose is to address and improve representation of CALD groups in clinical research. Significance for public healthMany people from culturally and linguistically diverse (CALD) backgrounds remain excluded from medical research such as clinical trials due to a range of language and cultural factors that can be amplified when this population is ageing. This exclusion has implications for the ability of CALD populations to benefit from participating in medical research and for applying research findings to CALD populations. It is essential to develop and implement strategies to include CALD communities in medical research and to uphold the ethical obligation of obtaining informed consent to research. The findings of this study have guided the development of a tablet-based resource which can be used in clinical and community contexts to raise awareness about the purpose of medical research. The resource has been carefully designed to be appropriate for participants' cultural background as well as their preferred language and literacy level. Such a resource has potential to address some of the cultural and linguistic barriers to clinical trial participation of CALD populations.
-
Item
-
ItemNo Preview AvailableA comparison of the discourse produced at different score levels of the OET writing sub-test(University of Melbourne, School of Languages & Linguistics, Language Testing Research Centre, 2013)
-
ItemNo Preview Available