讲座题目： Difference =Unfairness?: Task type, examinee background and factor structure of speaking tests for international teaching assistants (ITAs)
讲座嘉宾：Dr. Xun Yan
讲座时间：2017年12月19日（周二）上午10:00~11:30
讲座地点：紫金港校区东五201会议室
欢迎广大师生积极参加！
Abstract
Fairness is a central concern in the development, validation, and use of language tests. It is related to item writing, score use and interpretation, examinee rights, along with considerations of ethics and responsibilities in the profession of language testing (AERA, APA, NCME, 1999; Davies, 2004; ILTA, 2010). Fairness can be defined in multiple dimensions, ranging from psychometric quality in the narrowest sense (e.g., comparable scores across examinee groups) to social justice in the broadest sense (e.g., educational equality). Fairness assurance in the broad sense tends to go beyond the control of test developers, prompting them to focus more on narrowly-focused fairness investigations. However, these investigations can be equally challenging because, while subgroup score differences may imply test bias, it should be scrutinized through close examinations of examinee background, task type and test contexts.
In this talk, I will discuss the unique challenges and opportunities for fairness investigations on ITA speaking tests, a not-very-large-scale but common type of speaking assessment in US universities. To demonstrate these challenges and the importance of bias scrutinization, I will report a case study on a local ITA speaking test, where a multi-group confirmatory factor analysis was performed on 1,804 exams across three years. Results suggest that the factor structure of the test was not invariant across examinees from different L1 backgrounds. However, these differences do not entail test bias; instead, examinee background was confounded with construct-relevant variables such as language proficiency. Therefore, I argue that fairness as a psychometric quality should be situated within the context of test development and use. Subgroup differences do not necessarily signify unfairness; sometimes, they can become supporting evidence for test validity.
Xun Yan is an Assistant Professor of Linguistics, Second Language Acquisition and Teacher Education (SLATE), and Educational Psychology at University of Illinois at Urbana-Champaign, where he is also the coordinator of the English Placement Test (EPT). His current research interests include post-admission language assessments, scale development and validation, rater performance and training, assessment literacy for language teachers, and test score use in educational settings. He is currently a member of the Editorial Advisory Board for Language Testing. Yan’s work has been published in Language Testing, Assessing Writing, Journal of Second Language Writing, and Asia-Pacific Education Researchers. He is currently working on a co-authored book, titled Local Language Testing: Design, Implementation, and Development (Routledge).
