Please use this identifier to cite or link to this item:
Scopus Web of Science® Altmetric
Type: Theses
Title: Reverse engineering language test constructs for Messick’s value implications: a sociolinguistic approach
Author: Motteram, Johanna Mary
Issue Date: 2016
School/Discipline: School of Humanities
Abstract: Industrial, high volume, high stakes language testing has recently emerged, which annually assesses the language proficiency of tens of millions of people, worldwide. Important decisions are made using test results, so the tests and the scoring procedures should be theoretically sound. This study explores an aspect of Messick’s validity matrix which explicitly identifies the potential for validity threat where there are hidden social and cultural assumptions in the test construct. This facet of validity research and theorisation has been largely ignored by the language testing field. The problem of implicit assumptions in test constructs is approached through a theoretical framework which combines Bernstein’s model of the structuring of pedagogic discourse and the results of Systemic Functional Linguistics (SFL) informed text analysis. The framework allows connections between semantic and lexicogrammatical analysis of test responses and the social and cultural assumptions of the institution which evaluates that discourse. Analysis using SFL was pursued on the basis that the orientation to language which underpins the test construct under investigation, the International English Language Testing System (IELTS) General Training Version Written Module Task One (GTVWMT#1) is ill equipped to address questions of social and cultural assumptions implicit in texts, tasks, and test constructs. The IELTS GTVWMT#1 scoring criteria performance dimension Task Achievement, and its constituent elements ‘tone’ and ‘the requirements of the task’ were the particular focus of the study. The embodiment relationship between a test’s construct and the scoring criteria used to inform evaluation of test texts is an assumption of the study. Responses (n=54) to a single IELTS GTVWMT#1 prompt were collected from a diverse potential candidate population. The prompt demanded a friendly letter which communicated three main messages. The responses were double marked by trained IELTS raters. Quantitative analyses informed targeted qualitative analyses, including text analysis to identify similarities between groups of texts with respect to semantic constituency, realisation of critical semantic text elements, and Tenor. The study found that there are implicit expectations in the test construct related to social and cultural values and assumptions. These come from; 1. The particular task used in the study. There is potentially limited access to the domains of language use specified in this task. 2. The situational context of testing which has an impact on need for explicitness, on order of information, and on inclusion of detail. 3. Community expectations of thank you letters, particularly as they are entwined with notions of gracious behaviour. The major limitation of the study is the small sample size. Major recommendations include further work on the development of a model of text in social context to support language assessment, and consideration of the semantic constituency and potential for the impact of socially originating semantic variation in realisation of texts in high stakes language tests.
Advisor: Winser, William Nelson
Walsh, John
Mickan, Peter Frank
White, Peter Robert Rupert
Dissertation Note: Thesis (Ph.D.) -- University of Adelaide, School of Humanities, 2016.
Keywords: language testing
systemic functional linguistics
thankyou letters
assessing writing
Provenance: Copyright material removed from digital thesis. See print copy in University of Adelaide Library for full text.
This electronic version is made publicly available by the University of Adelaide in accordance with its open access policy for student theses. Copyright in this thesis remains with the author. This thesis may incorporate third party material which has been used by the author pursuant to Fair Dealing exceptions. If you are the owner of any included third party copyright material you wish to be removed from this electronic version, please complete the take down form located at:
DOI: 10.25909/5b3d80e347ffe
Appears in Collections:Research Theses

Files in This Item:
File Description SizeFormat 
01front.pdf108.25 kBAdobe PDFView/Open
02whole.pdf8.11 MBAdobe PDFView/Open
PermissionsLibrary staff access only374.08 kBAdobe PDFView/Open
RestrictedLibrary staff access only9.61 MBAdobe PDFView/Open

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.