The present study aims to meta-analyze the reliability of second language (L2) reading assessments and identify the potential moderators of reliability in L2 reading comprehension tests. We examined 3,247 individual studies for possible inclusion and assessed 353 studies as eligible for the inclusion criteria. Of these, we extracted 150 Cronbach’s alpha estimates from 113 eligible studies (years 1998–2024) that reported Cronbach’s alpha coefficients properly and coded 27 potential predictors comprising of the characteristics of the study, the test, and test takers. We subsequently conducted a reliability generalization (RG) meta-analysis to compute the average reliability coefficient of L2 reading comprehension tests and identify potential moderators from 27 coded predictor variables. The RG meta-analysis found an average reliability of 0.79 (95% CI [0.78, 0.81]). The number of test items, test piloting, test takers’ educational institution, study design, and testing mode were found to respectively explain 16.76%, 5.92%, 4.91%, 2.58%, and 1.36% of variance in reliability coefficients. The implications of this study and future directions are further discussed.

中文翻译：

---

第二语言阅读理解评估工具可靠性的荟萃分析


本研究旨在对第二语言 （L2） 阅读评估的可靠性进行荟萃分析，并确定 L2 阅读理解测试中可靠性的潜在调节因素。我们检查了 3,247 项可能纳入的单独研究，并评估了 353 项研究符合纳入标准。其中，我们从 113 项符合条件的研究（1998-2024 年）中提取了 150 个 Cronbach 的 alpha 估计值，这些研究正确报告了 Cronbach 的 alpha 系数，并编码了 27 个潜在预测因子，包括研究、测试和应试者的特征。随后，我们进行了可靠性泛化 （RG） 荟萃分析，以计算 L2 阅读理解测试的平均信度系数，并从 27 个编码预测变量中确定潜在的调节因子。RG meta分析发现，平均信度为0.79（95% CI [0.78， 0.81]）。发现测试项目数量、测试试点、应试者教育机构、学习设计和测试模式分别解释了 16.76% 、 5.92% 、 4.91% 、 2.58% 和 1.36% 的信度系数方差。进一步讨论了本研究的意义和未来的方向。