TY - JOUR ID - 15417 TI - Interpreting the Validity of a High-Stakes Test in Light of the Argument-Based Framework: Implications for Test Improvement JO - Journal of Research in Applied Linguistics JA - RALS LA - en SN - 2345-3303 AU - Darabi Bazvand, Ali AU - Ahmadi, Alireza AD - Department of English Language, College of Languages , University of Human Developmentv, Sulaimani, Kurdistan, Iraq AD - Shiraz University Y1 - 2020 PY - 2020 VL - 11 IS - 1 SP - 66 EP - 88 KW - Argument-Based Validity KW - Differential Item Functioning (DIF) KW - ELT DO - 10.22055/rals.2020.15417 N2 - The validity of large-scale assessments may be compromised, partly due to their content inappropriateness or construct underrepresentation. Few validity studies have focused on such assessments within an argument-based framework. This study analyzed the domain description and evaluation inference of the Ph.D. Entrance Exam of ELT (PEEE) sat by Ph.D. examinees (n = 999) in 2014 in Iran. To track evidence for domain definition, the test content was scrutinized by applied linguistics experts (n = 12). As for evaluation inference, the reliability and differential item functioning (DIF) of the test were examined. Results indicated that the test is biased because (1) the test tasks are not fully represented in the Ph.D. course objectives, (2) the test is best reliable for high-ability test-takers (IRT analysis), and (3) 4 items are flagged for nonnegligible DIF (logistic regression [LR] analysis). Implications for language testing and assessment are discussed and some possible suggestions are offered. UR - https://rals.scu.ac.ir/article_15417.html L1 - https://rals.scu.ac.ir/article_15417_290f49e78050fb15453cbedd89f9197f.pdf ER -