Infant and Child Development ( IF 2.8 ) Pub Date : 2024-04-16 , DOI: 10.1002/icd.2506 Pamela E. Davis‐Kean 1 , Alexa Ellis 2 , Moin Syed 3
Events of the past decade have revealed substantial limitations in our standard approach to evaluating manuscripts for publication. Preference for ‘positive results’ and findings that are surprising or novel has led to a substantial publication bias that casts doubt on large portions of the existing literature (Davis-Kean & Ellis, 2019; Scheel et al., 2021). Registered Reports represent a major initiative to combat these problems, as they shift the focus of evaluation from the nature of the findings to the strength of the conceptualization, research design and analytic plan (Chambers, 2013).
Contrary to the standard review process, with Registered Reports the process is split into two distinct stages. Authors initially submit a Stage 1 proposal consisting of the Introduction, Method and Planned Analysis sections prior to conducting the study. The Stage 1 proposal is sent for peer review, with an ultimate positive outcome of an ‘in principle acceptance,’ which is a guarantee that the journal will publish the full article, regardless of the results, providing the authors conduct the study as planned and do so competently. Following the in principle acceptance authors collect the data and/or conduct the analysis and then submit the Stage 2 report for final review. For more information, and answers to frequently asked questions about Registered Reports, see https://cos.io/rr/.
Registered Reports have been increasingly adopted in journals across the sciences in general, and psychology in particular. Although uptake had initially been slow amongst developmental journals, this has changed considerably in recent years (see Syed et al., 2023, for Registered Reports specifically and Silverstein et al., 2024, for open science and metascience more generally). Nevertheless, there remain many questions about how the format works for complex longitudinal designs and secondary data (van den Akker et al., 2021), both of which are common in developmental research (see also Syed & Donnellan, 2020).
The purpose of this Special Issue was to feature Registered Reports using secondary (pre-existing) data pertaining to developmental issues from the prenatal period through early adulthood. Secondary datasets refer to data collected by someone other than the primary user. Datasets used in the eight articles featured in the special issue covered wide ground. Two of the articles relied on the National Institute of Child Health and Human Development Study of Early Child Care and Youth Development, one to investigate how early caregiver interactions are related to educational attainment, income and employment (Duncan et al., 2024), and the other to compare relations between parental sensitivity and two different methods for assessing attachment (Nivison et al., 2024). Examining parenting from a different perspective, Wright and Jackson (2024) drew from the German Socioeconomic Panel study to test the independent predictive power of parent and child personality traits on a variety of adolescent outcomes (e.g., health, education, family and civic engagement). Several studies focused on the schooling context: Spiegler et al. (2024) used the Children of Immigrants Longitudinal Survey in Four European Countries to examine how peer victimisation was related to classroom ethnic diversity and teacher interactions; Kim and Sidney (2024) used the Trends in International Mathematics and Science Study to examine sources of influences for individual differences in students' academic self-concept; and Zimmermann et al. (2024) used the Early Childhood Longitudinal Study Kindergarten class of 2011 to understand how pre-Kindergarten enrollment was related to indicators of social development across the transition into and out of Kindergarten. Finally, two studies examined physiological aspects of development: Chaku and Barry (2024) used the Adolescent Brain Cognitive Development study to understand how hormonal profiles were associated with cognitive development, and Woods et al. (2024) drew from the Future of Families and Child Wellbeing Study to test whether indicators of sleep were longitudinally associated with cognitive and behavioural development.
Taken together, the articles in the current Special Issue highlight both the feasibility and utility of Registered Reports with secondary developmental data. It is abundantly clear from the papers published for this special issue that Registered Reports are a viable option for those who are using existing or secondary datasets, and that they only scratch the surface of available datasets that could be put to good use with Registered Reports. Some other examples of secondary datasets include but are not limited to Monitoring the Future, Panel Study of Income Dynamics, National Longitudinal Surveys, British Cohort Studies, Growing Up in Scotland and many more (see Davis-Kean & Ellis, 2019, for a review, and Davis-Kean et al., 2015, for more details on how to access these and other datasets). All of these datasets, and many more, are potentially suitable for Registered Reports but are underused with the format.
We do, however, want to highlight a few important issues that were noted during the review process. First, reviewers sometimes struggle with how to review papers that do not contain data, but only the ‘proposal’ of the idea and methods. For secondary data in particular, reviewers struggled with the limitation of existing data already having been collected and not being able to advise on new primary data collection. Often, as has been noted by Davis-Kean et al. (2015), there are limitations on what questions can be answered with secondary data. There is no option to add data to the existing data or change the methods of the study. Reviewers would often suggest such changes for these papers that were not possible. Thus, a recommendation for future secondary data Registered Reports is that very clear reviewer information be provided on the role of the reviewer but also the limitations of what can be requested by the reviewer to the authors. Basically, the methodology and data structure are generally already known for the study, the reviewers' job is to determine if the questions are of interest to the field of study and if the methods and the data set being used are suitable to answer the questions being asked.
A challenge that exists in the interaction between the reviewers and authors, evident for both preregistrations and Registered Reports, is the frequent need for robustness checks that may not have been previously registered, but are important for validating the findings. These are often seen as exploratory analyses but often are reasonable requests for checking the validity of a finding when a reviewer notes an alternative hypothesis that is both plausible and testable in the data set. Unlike primary data collection which may be restricted in the breadth of variables available to do the robustness check, secondary data will often have the available constructs that could be tested. We recommend authors and reviewers think through the addition of alternative specifications or hypotheses that could be tested with the existing data at the Stage 1 proposal phase. Sometimes, however, such realisations occur after the fact and thus must be added as exploratory analyses to the Stage 2 manuscript. Authors are encouraged to do this, and editors and reviewers are encouraged to facilitate it, so long as the additions are transparently reported, as we always want to be sure we are conducting the most rigorous tests in our studies.
Finally, on the initial submission for review for this special issue, it was sometimes the case that the actual questions being submitted were not major questions of the field, or could not adequately be tested with the proposed dataset. Any analysis of secondary data could be done, but it is important for us to all consider whether it should be done. One of the primary goals of Registered Reports is to ensure that a study is informative of its research questions regardless of the results. This design feature demands high-quality studies and is particularly well-suited for the most complex, controversial and transformative research because it provides the clearest constraints on the questionable research practices and publication bias that often plague developmental research (Davis-Kean & Ellis, 2019). Elevating this format to being one where having authorship on a Registered Report denotes that extreme care and careful planning were done to obtain the finding will help to increase the confidence in the research that is done on infant and child development.
中文翻译:
二次发育数据注册报告:特刊简介
过去十年的事件揭示了我们评估出版手稿的标准方法存在很大的局限性。对“积极结果”和令人惊讶或新颖的发现的偏爱导致了严重的出版偏见,使人们对大部分现有文献产生了怀疑(Davis-Kean & Ellis, 2019;Scheel 等, 2021)。注册报告代表了解决这些问题的一项重大举措,因为它们将评估的重点从研究结果的性质转移到概念化、研究设计和分析计划的强度上(Chambers, 2013)。
与标准审查流程相反,注册报告的流程分为两个不同的阶段。在进行研究之前,作者首先提交由简介、方法和计划分析部分组成的第一阶段提案。第一阶段提案将送去同行评审,最终获得“原则上接受”的积极结果,这保证了期刊将发表完整的文章,无论结果如何,前提是作者按计划进行研究,并且胜任地这样做。在原则上接受之后,作者收集数据和/或进行分析,然后提交第二阶段报告以供最终审查。有关注册报告的更多信息和常见问题解答,请参阅 https://cos.io/rr/。
注册报告已越来越多地被科学期刊(尤其是心理学期刊)采用。尽管发展期刊的吸收最初缓慢,但近年来这种情况发生了很大变化(具体参见 Syed et al., 2023,针对注册报告,以及 Silverstein et al., 2024,针对更广泛的开放科学和元科学)。然而,关于该格式如何适用于复杂的纵向设计和二手数据(van den Akker et al., 2021),仍然存在许多问题,这两者在发展研究中都很常见(另请参见 Syed & Donnellan, 2020)。
本特刊的目的是使用与从产前时期到成年早期的发育问题有关的二手(预先存在的)数据来展示注册报告。次要数据集是指由主要用户以外的其他人收集的数据。该特刊中的八篇文章中使用的数据集涵盖了广泛的领域。其中两篇文章依赖于国家儿童健康和人类发展研究所的早期儿童保育和青少年发展研究,其中一篇研究了早期看护者互动与教育程度、收入和就业之间的关系(Duncan 等人, 2024),以及另一个是比较父母敏感性与两种不同的依恋评估方法之间的关系(Nivison 等人, 2024)。 Wright 和 Jackson( 2024 )从不同的角度审视养育方式,借鉴了德国社会经济小组的研究,测试了父母和孩子的人格特征对各种青少年结果(例如健康、教育、家庭和公民参与)的独立预测能力。 。几项研究重点关注学校教育背景:Spiegler 等人。 (2024)利用四个欧洲国家的移民儿童纵向调查来研究同伴受害与课堂种族多样性和教师互动之间的关系; Kim和Sidney(2024)利用国际数学和科学研究趋势来考察学生学业自我概念个体差异的影响来源;和齐默尔曼等人。 ( 2024 ) 使用 2011 年幼儿园班级的早期儿童纵向研究来了解学前班入学率与进入和离开幼儿园期间的社会发展指标之间的关系。最后,两项研究考察了发育的生理方面:Chaku 和 Barry ( 2024 ) 使用青少年大脑认知发育研究来了解荷尔蒙分布如何与认知发育相关,而 Woods 等人。 ( 2024 ) 根据家庭未来和儿童福祉研究来测试睡眠指标是否与认知和行为发展纵向相关。
总而言之,当前特刊中的文章强调了带有二次开发数据的注册报告的可行性和实用性。从本期特刊发表的论文中可以清楚地看出,对于那些使用现有或辅助数据集的人来说,注册报告是一个可行的选择,并且它们仅触及了可以充分利用注册报告的可用数据集的表面。二级数据集的其他一些示例包括但不限于“监测未来”、“收入动态小组研究”、“国家纵向调查”、“英国队列研究”、“在苏格兰长大”等等(请参阅 Davis-Kean & Ellis, 2019 年的评论) ,以及 Davis-Kean 等人, 2015,了解有关如何访问这些和其他数据集的更多详细信息)。所有这些数据集以及更多数据集都可能适合注册报告,但该格式未得到充分利用。
然而,我们确实想强调审查过程中注意到的一些重要问题。首先,审稿人有时会为如何审阅不包含数据而仅包含想法和方法的“建议”的论文而苦苦挣扎。特别是对于二手数据,审稿人面临着已经收集的现有数据的限制,并且无法就新的主要数据收集提供建议。通常,正如戴维斯-基恩等人所指出的那样。 (2015),二手数据可以回答哪些问题是有限的。无法选择向现有数据添加数据或更改研究方法。审稿人经常会建议对这些论文进行这样的修改,但这是不可能的。因此,对未来二手数据注册报告的建议是,提供关于审稿人角色的非常清晰的审稿人信息,但也提供审稿人向作者提出的要求的限制。基本上,研究的方法和数据结构通常是已知的,审稿人的工作是确定问题是否对研究领域感兴趣,以及所使用的方法和数据集是否适合回答正在研究的问题。问道。
审稿人和作者之间的互动中存在的一个挑战,对于预注册和注册报告来说都是显而易见的,那就是经常需要进行稳健性检查,这些检查可能以前没有注册过,但对于验证结果很重要。这些通常被视为探索性分析,但当审阅者注意到数据集中合理且可检验的替代假设时,通常是检查结果有效性的合理要求。与可用于进行稳健性检查的变量广度可能受到限制的主要数据收集不同,辅助数据通常具有可测试的可用构造。我们建议作者和审稿人考虑添加替代规范或假设,这些规范或假设可以在第一阶段提案阶段使用现有数据进行测试。然而,有时这种认识是在事后发生的,因此必须作为探索性分析添加到第二阶段手稿中。我们鼓励作者这样做,也鼓励编辑和审稿人为其提供便利,只要透明地报告添加的内容,因为我们始终希望确保我们在研究中进行最严格的测试。
最后,在最初提交该特刊供审查时,有时会出现这样的情况:提交的实际问题不是该领域的主要问题,或者无法使用提议的数据集进行充分测试。对二手数据的任何分析都可以进行,但重要的是我们所有人都考虑是否应该这样做。注册报告的主要目标之一是确保研究能够提供其研究问题的信息,无论结果如何。这种设计功能需要高质量的研究,特别适合最复杂、有争议和变革性的研究,因为它对经常困扰发展研究的可疑研究实践和出版偏见提供了最明确的限制(Davis-Kean & Ellis, 2019))。将这种格式提升为在注册报告上拥有作者身份的格式,表明为获得研究结果而付出了极大的谨慎和仔细的计划,这将有助于增加对婴儿和儿童发育研究的信心。