When used appropriately, non-significant p-values have the potential to further our understanding of what does not work in education, and why. When misinterpreted, they can trigger misguided conclusions, for example about the absence of an effect of an educational intervention, or about a difference in the efficacy of different interventions. We examined the frequency of non-significant p-values in recent volumes of peer-reviewed educational research journals. We also examined how frequently researchers misinterpret non-significance to imply the absence of an effect, or a difference to another significant effect. Within a random sample of 50 peer-reviewed articles, we found that of 528 statistically tested hypotheses, 253 (48%) were non-significant. Of these, 142 (56%) were erroneously interpreted to indicate the absence of an effect, and 59 (23%) to indicate a difference to another significant effect. For 97 (38%) of non-significant results, such misinterpretations were linked to potentially misguided implications for educational theory, practice, or policy. We outline valid ways for dealing with non-significant p-values to improve their utility for education, discussing potential reasons for these misinterpretations and implications for research.

中文翻译：

---

提高非显著性结果在教育研究中的作用：综述和建议


如果使用得当，不显著的 p 值有可能进一步加深我们对教育中不起作用的内容以及原因的理解。当被误解时，它们可能会引发误导性的结论，例如关于教育干预没有效果，或者关于不同干预效果的差异。我们检查了最近几卷同行评审的教育研究期刊中不显著的 p 值的频率。我们还研究了研究人员将 non-significance 误解为暗示没有效果或与另一个显著性效果不同的频率。在 50 篇同行评议文章的随机样本中，我们发现在 528 个经过统计检验的假设中，有 253 个 （48%） 是不显著的。其中，142 项 （56%） 被错误地解释为表示没有效果，59 项 （23%） 表示与另一种显著效果存在差异。对于 97 个 （38%） 的非显著性结果，此类误解与对教育理论、实践或政策的潜在误导性影响有关。我们概述了处理不显著的 p 值以提高其教育效用的有效方法，讨论了这些误解的潜在原因和对研究的影响。