School of Languages and Linguistics - Theses
Permanent URI for this collection
Search Results
1 - 2 of 2
-
ItemThe process of the assessment of writing performance: the rater's perspective( 2000)The primary purpose of this study is to investigate the process by which raters of texts written by ESL learners make their scoring decisions. The context is the Special Test of English Proficiency (step), used by the Australian government to assist in immigration decisions. Four trained, experienced and reliable step raters took part in the study, providing scores for two sets of 24 texts. The first set was scored as in an operational rating session. Raters then provided think-aloud protocols describing the rating process as they rated the second set. Scores were compared under the two conditions and comparisons made with the raters' operational rating behaviour. Both similarities and differences were observed. A coding scheme developed to describe the think-aloud data allowed analysis of the sequence of rating, the interpretations the raters made of the scoring categories in the analytic rating scale, and the difficulties raters faced in rating. Findings demonstrate that raters follow a fundamentally similar rating process, in three stages. With some exceptions, they appear to hold similar interpretations of the scale categories and descriptors, but the relationship between scale contents and text quality remains obscure. A model is presented describing the rating process. This shows that rating is at one level a rule-bound, socially governed procedure that relies upon a rating scale and the rater training which supports it, but it retains an indeterminate component as a result of the complexity of raters' reactions to individual texts. The task raters face is to reconcile their impression of the text, the specific features of the text, and the wordings of the rating scale, thereby producing a set of scores. The rules and the scale do not cover all eventualities, forcing the raters to develop various strategies to help them cope with problematic aspects of the rating process. In doing this they try to remain close to the scale, but are also heavily influenced by the complex intuitive impression of the text obtained when they first read it. This sets up a tension between the rules and the intuitive impression, which raters resolve by what is ultimately a somewhat indeterminate process. In spite of this tension and indeterminacy, rating can succeed in yielding consistent scores provided raters are supported by adequate training, with additional guidelines to assist them in dealing with problems. Rating requires such constraining procedures to produce reliable measurement.
-
ItemThe nature and conditions of pragmatic and discourse transfer in cross-cultural interaction investigated through naturalized role-play( 2004)For decades, the first linguistic and cultural influence on second language performance (technically known as pragmatic and discourse transfer) in cross-cultural interaction has fascinated researchers because its nature and especially its conditions have never been fathomed out. The aims of this investigation are threefold. First, it examines the nature of pragmatic and discourse transfer in compliment responses by Vietnamese speakers of English as a second language in cross-cultural interaction with Australians. The examination also takes into account data from conversations among Australian English native speakers and interaction between Vietnamese counterparts. Second, the research project investigates the underexplored conditions of pragmatic and discourse transfer. In the quest for the nature and conditions of pragmatic and discourse transfer, research methodologies provoke much debate because they have different advantages and disadvantages, though the ultimate goal remains the controlled elicitation of data that is comparable to real-life production. The third aim of the present study is to propose and validate an innovative methodology of data collection in cross-cultural and interlanguage pragmatics research the Naturalized Role-play. This methodology is capable of realizing the highly desirable but virtually impossible goal of eliciting spontaneous data in controlled settings. In reference to the methodological design of the research project, the Naturalized Role-play provided the main corpus of data on pragmatic and discourse transfer whose in-depth analysis revealed the nature of this phenomenon. In addition to Naturalized Role-play data, background questionnaire and retrospective interview data was collected to explore conditions of pragmatic and discourse transfer. To demonstrate the effectiveness and validity of the Naturalized Role-play, compliment response data collected by means of the Naturalized Role-play was compared with data from other major methods including the questionnaire, closed role-play, open role-play and natural data recording. Findings of this investigation indicated what was transferred and how pragmatic and discourse transfer patterned, upon which new hypotheses were formulated. The investigation also uncovered as yet unknown conditions of pragmatic and discourse transfer (e.g. awareness in language production) and their interaction. Moreover, the Naturalized Role-play proved to fulfil its aim. It can combine advantages of major methods without their drawbacks and is more effective (i.e. providing more natural data) than existing means of data elicitation. In essence, based on the Naturalized Role-play approach, this investigation sheds new light on the nature of pragmatic and discourse transfer, offers insights into its conditions and features a pioneering creative solution to the controversial methodological problem. The study also presents implications of its findings for second language learners, teachers and native speakers of different languages in social interactions where cultures meet.