Educational Research Review ( IF 9.6 ) Pub Date : 2023-12-30 , DOI: 10.1016/j.edurev.2023.100591 Huanyou Chai , Tianhui Hu , Li Wu
Given the widespread concern on collaborative problem solving (CPS) skills, there has been an increasing interest in the last few years to explore how to assess them with digital technologies. This study systematically reviewed how CPS skills have been assessed with digital technologies in the literature. A total of 40 articles were reviewed to analyze specific computer-based assessment instruments of CPS skills from four perspectives: research context, theoretical model for assessment, assessment type, and reliability and validity evidence. The results indicate that most tests target a sample of less than 500 junior students. Nine theoretical models are employed for assessing CPS skills, most of which treat these skills as an explicit combination of social and cognitive skills and are applied to a limited range of participants' age levels, collaboration features, and team compositions. A total of 22 tests have been employed and fallen into four types, i.e., the ones with specific predefined messages in human-agent mode, and those with online chat box, videoconferencing, and face-to-face collaboration in human-human mode. Each type of these tests demonstrates great diversities in participants’ age levels, types of CPS task(s), team compositions, types of assessment data, and methods of data recording and scoring. A certain number of tests lack reliability and validity evidence. Our findings are expected to benefit relevant researchers and test developers in terms of providing suggestions for future research which include testing the applicability of theoretical models for assessing CPS skills across a wide range of assessment contexts. In addition, future researchers should improve the development, data processing, and report of those four types of computer-based assessment instruments of CPS skills through different approaches, respectively.
中文翻译:
基于计算机的协作解决问题技能评估:实证研究的系统回顾
鉴于人们对协作问题解决(CPS)技能的广泛关注,过去几年人们越来越有兴趣探索如何使用数字技术评估这些技能。本研究系统回顾了文献中如何使用数字技术评估 CPS 技能。共回顾了 40 篇文章,从研究背景、评估理论模型、评估类型以及信度和效度证据四个角度分析了具体的基于计算机的 CPS 技能评估工具。结果表明,大多数测试针对的样本不足 500 名初中学生。采用九种理论模型来评估 CPS 技能,其中大多数将这些技能视为社交和认知技能的明确组合,并应用于有限范围的参与者年龄水平、协作特征和团队组成。总共使用了22个测试,分为四种类型,即人与代理模式下的特定预定义消息的测试,以及人与人模式下的在线聊天框、视频会议和面对面协作的测试。每种类型的测试都显示出参与者的年龄水平、CPS 任务类型、团队组成、评估数据类型以及数据记录和评分方法的巨大差异。某些测试缺乏可靠性和有效性证据。我们的研究结果预计将使相关研究人员和测试开发人员受益,为未来的研究提供建议,其中包括测试理论模型在广泛的评估环境中评估 CPS 技能的适用性。 此外,未来的研究人员应该通过不同的方法分别改进这四种基于计算机的CPS技能评估工具的开发、数据处理和报告。