Background

A recent review found that 45% of meta-analyses included statistical errors, of which, the most common was the calculation of effect sizes based on standard error (SE) rather than standard deviation (SD) [the SE/SD mix-up].

Objectives

The first aim of this study was to assess the impact of the SE/SD mix-up on the results of one highly cited meta-analysis. Our second aim was to identify one potential source of the SE/SD mix-up, by assessing how often SE is reported as a measure of sample variability in randomised controlled trials in sports medicine.

Methods

We checked for potential SE/SD mix-ups in a 2015 meta-analysis of randomised controlled trials reporting the effects of recreational football interventions on aerobic fitness in adults. We corrected effect sizes affected by SE/SD mix-ups and re-analysed the data according to the original methodology. We compared pooled estimates of effect sizes from our re-analysis of corrected values with those of the original study. To assess how often SE was reported instead of SD as a measure of sample variance, we text mined results of randomised controlled trials from seven sports medicine journals and reported the proportion reporting of SE versus SD.

Results

We identified potential SE/SD mix-ups in 9/16 effect sizes included in the meta-analysis describing the effects of football-based interventions versus non-exercise control. The published effect size was standardised mean difference (SMD) = 1.46 (95% confidence interval [CI] 0.91, 2.01). After correcting for SE/SD mix-ups, our re-analysis produced a smaller pooled estimate (SMD = 0.54 [95% CI 0.37, 0.71]). The original pooled estimate for trials comparing football versus running interventions was SMD = 0.68 (95% CI 0.06, 1.4). After correcting for SE/SD mix-ups and re-analysis, the effect was no longer statistically significant (SMD = 0.20 [95% CI − 0.10, 0.49)]). We found that 19.3% of randomised controlled trials reported SE rather than SD to describe sample variability. The relative frequency of the practice ranged from 0 to 25% across the seven journals sampled.

Conclusions

We found the SE/SD mix-up had inflated estimates for the effects of football on aerobic fitness. Meta-analysts should be vigilant to avoid miscalculating effect sizes. Authors, reviewers and editors should avoid and discourage (respectively) the practice of reporting SE as a measure of sample variability in sports medicine research.

中文翻译：

---

标准误差/标准差混淆：对运动医学荟萃分析的潜在影响

背景

最近的一项审查发现，45% 的荟萃分析包含统计误差，其中最常见的是基于标准误差 (SE) 而不是标准差 (SD) 计算效应大小 [SE/SD 混合] 。

目标

本研究的第一个目的是评估 SE/SD 混合对一项被高度引用的荟萃分析结果的影响。我们的第二个目标是通过评估运动医学随机对照试验中将 SE 报告为样本变异性衡量标准的频率，来确定 SE/SD 混淆的一个潜在来源。

方法

我们在 2015 年随机对照试验的荟萃分析中检查了潜在的 SE/SD 混淆，该试验报告了休闲性足球干预措施对成人有氧健身的影响。我们纠正了 SE/SD 混合影响的效应大小，并根据原始方法重新分析了数据。我们将校正值重新分析得出的效应大小汇总估计值与原始研究的效应值估计值进行了比较。为了评估报告 SE 而不是 SD 作为样本方差度量的频率，我们对七种运动医学期刊的随机对照试验结果进行文本挖掘，并报告了 SE 与 SD 的报告比例。

结果

我们在描述足球干预措施与非运动控制效果的荟萃分析中发现了 9/16 效应大小中潜在的 SE/SD 混淆。公布的效应大小为标准化平均差 (SMD) = 1.46（95% 置信区间 [CI] 0.91, 2.01）。校正 SE/SD 混淆后，我们的重新分析产生了较小的汇总估计值 (SMD = 0.54 [95% CI 0.37, 0.71])。比较足球与跑步干预措施的试验的原始汇总估计值为 SMD = 0.68（95% CI 0.06，1.4）。校正 SE/SD 混淆并重新分析后，效果不再具有统计显着性（SMD = 0.20 [95% CI − 0.10, 0.49)]）。我们发现 19.3% 的随机对照试验报告 SE 而不是 SD 来描述样本变异性。在七种抽样期刊中，这种做法的相对频率范围为 0 到 25%。

结论

我们发现 SE/SD 的混淆夸大了足球对有氧健身影响的估计。荟萃分析师应保持警惕，避免错误计算效应大小。作者、审稿人和编辑应（分别）避​​免和劝阻报告 SE 作为运动医学研究中样本变异性的衡量标准。