Systematic reviews are a time-consuming yet effective approach to understanding research trends. While researchers have investigated how to speed up the process of screening studies for potential inclusion, few have focused on to what extent we can use algorithms to extract data instead of human coders. In this study, we explore to what extent analyses and algorithms can produce results similar to human data extraction during a scoping review—a type of systematic review aimed at understanding the nature of the field rather than the efficacy of an intervention—in the context of a never before analyzed sample of studies that were intended for a scoping review. Specifically, we tested five approaches: bibliometric analysis with VOSviewer, latent Dirichlet allocation (LDA) with bag of words, k-means clustering with TF-IDF, Sentence-BERT, or SPECTER, hierarchical clustering with Sentence-BERT, and BERTopic. Our results showed that topic modeling approaches (LDA/BERTopic) and k-means clustering identified specific, but often narrow research areas, leaving a substantial portion of the sample unclassified or in unclear topics. Meanwhile, bibliometric analysis and hierarchical clustering with SBERT were more informative for our purposes, identifying key author networks and categorizing studies into distinct themes as well as reflecting the relationships between themes, respectively. Overall, we highlight the capabilities and limitations of each method and discuss how these techniques can complement traditional human data extraction methods. We conclude that the analyses tested here likely cannot fully replace human data extraction in scoping reviews but serve as valuable supplements.

系统评价是了解研究趋势的一种耗时但有效的方法。虽然研究人员已经研究了如何加快筛选研究以纳入潜在数据的过程，但很少有人关注我们可以在多大程度上使用算法而不是人工编码人员来提取数据。在这项研究中，我们探讨了分析和算法在范围界定审查期间可以在多大程度上产生类似于人类数据提取的结果——一种旨在了解该领域性质而不是干预有效性的系统审查——在以前从未分析过的研究样本的背景下，旨在进行范围审查。具体来说，我们测试了五种方法：使用 VOSviewer 进行文献计量分析，使用词袋进行潜在狄利克雷分配 （LDA），使用 TF-IDF、Sentence-BERT 或 SPECTER 进行 k-means 聚类，使用 Sentence-BERT 和 BERTopic 进行分层聚类。我们的结果表明，主题建模方法 （LDA/BERTopic） 和 k-means 聚类确定了特定但通常狭窄的研究领域，使样本的很大一部分未分类或不明确主题。同时，使用 SBERT 进行文献计量分析和分层聚类对我们的目的来说信息量更大，可以识别关键作者网络并将研究分类为不同的主题，并分别反映主题之间的关系。总的来说，我们强调了每种方法的功能和局限性，并讨论了这些技术如何补充传统的人类数据提取方法。我们得出的结论是，这里测试的分析可能无法完全取代范围综述中的人类数据提取，但可以作为有价值的补充。