Technological aids are ubiquitous in today's educational environments. Whereas much of the dust has settled in the debate on how to validate traditional educational solutions, in the area of technology-enhanced learning (TEL) many questions still remain. Technologies often abstract away student behaviour by condensing actions into numbers, meaning teachers have to assess student data rather than observing students directly. With the rapid adoption of artificial intelligence in education, it is timely to obtain a clear image of the landscape of validity criteria relevant to TEL. In this paper, we conduct a systematic review of research on TEL interventions, where we combine active learning for title and abstract screening with a backward snowballing phase. We extract information on the validity criteria used to evaluate TEL solutions, along with the methods employed to measure these criteria. By combining data on the research methods (qualitative versus quantitative) and knowledge source (theory versus practice) used to inform validity criteria, we ground our results epistemologically. We find that validity criteria tend to be assessed more positively when quantitative methods are used and that validation framework usage is both rare and fragmented. Yet, we also find that the prevalence of different validity criteria and the research methods used to assess them are relatively stable over time, implying that a strong foundation exists to design holistic validation frameworks with the potential to become commonplace in TEL research.

中文翻译：

---

了解技术增强学习的有效性标准：系统文献综述


技术辅助工具在当今的教育环境中无处不在。尽管关于如何验证传统教育解决方案的争论已经尘埃落定，但在技术增强学习 (TEL) 领域仍然存在许多问题。技术通常通过将行为压缩为数字来抽象学生的行为，这意味着教师必须评估学生的数据，而不是直接观察学生。随着人工智能在教育领域的快速应用，我们需要及时了解与 TEL 相关的有效性标准的情况。在本文中，我们对 TEL 干预研究进行了系统回顾，其中我们将标题和摘要筛选的主动学习与向后滚雪球阶段相结合。我们提取有关用于评估 TEL 解决方案的有效性标准的信息，以及用于衡量这些标准的方法。通过结合用于告知有效性标准的研究方法（定性与定量）和知识来源（理论与实践）的数据，我们在认识论上奠定了我们的结果。我们发现，当使用定量方法时，有效性标准往往会得到更积极的评估，并且验证框架的使用既罕见又分散。然而，我们还发现，不同有效性标准的普遍性以及用于评估它们的研究方法随着时间的推移相对稳定，这意味着设计整体验证框架具有坚实的基础，有可能在 TEL 研究中变得普遍。