Simulation studies are widely used for evaluating the performance of statistical methods in psychology. However, the quality of simulation studies can vary widely in terms of their design, execution, and reporting. In order to assess the quality of typical simulation studies in psychology, we reviewed 321 articles published in Psychological Methods, Behavior Research Methods, and Multivariate Behavioral Research in 2021 and 2022, among which 100/321 = 31.2% report a simulation study. We find that many articles do not provide complete and transparent information about key aspects of the study, such as justifications for the number of simulation repetitions, Monte Carlo uncertainty estimates, or code and data to reproduce the simulation studies. To address this problem, we provide a summary of the ADEMP (aims, data-generating mechanism, estimands and other targets, methods, performance measures) design and reporting framework from Morris et al. (2019) adapted to simulation studies in psychology. Based on this framework, we provide ADEMP-PreReg, a step-by-step template for researchers to use when designing, potentially preregistering, and reporting their simulation studies. We give formulae for estimating common performance measures, their Monte Carlo standard errors, and for calculating the number of simulation repetitions to achieve a desired Monte Carlo standard error. Finally, we give a detailed tutorial on how to apply the ADEMP framework in practice using an example simulation study on the evaluation of methods for the analysis of pre-post measurement experiments. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

中文翻译：

---

心理学方法论研究的模拟研究：用于规划、预注册和报告的标准化模板。


模拟研究广泛用于评估心理学中统计方法的性能。但是，仿真研究的质量在设计、执行和报告方面可能差异很大。为了评估心理学典型模拟研究的质量，我们回顾了 2021 年和 2022 年发表在 Psychological Methods、Behavior Research Methods 和 Multivariate Behavioral Research 上的 321 篇文章，其中 100/321 = 31.2% 报告了模拟研究。我们发现，许多文章没有提供有关研究关键方面的完整和透明的信息，例如模拟重复次数的证明、蒙特卡洛不确定性估计或重现模拟研究的代码和数据。为了解决这个问题，我们提供了 Morris 等人（2019 年）的 ADEMP（目标、数据生成机制、估计量和其他目标、方法、绩效衡量标准）设计和报告框架的摘要，该框架适用于心理学的模拟研究。基于此框架，我们提供了 ADEMP-PreReg，这是一个分步模板，供研究人员在设计、预注册和报告他们的模拟研究时使用。我们给出了用于估计常见性能度量、它们的 Monte Carlo 标准误差以及计算模拟重复次数以实现所需 Monte Carlo 标准误差的公式。最后，我们通过一个关于前后测量实验分析方法评估的示例模拟研究，给出了有关如何在实践中应用 ADEMP 框架的详细教程。（PsycInfo 数据库记录 （c） 2024 APA，保留所有权利）。