In microbiome disease association studies, it is a fundamental task to test which microbes differ in their abundance between groups. Yet, consensus on suitable or optimal statistical methods for differential abundance testing is lacking, and it remains unexplored how these cope with confounding. Previous differential abundance benchmarks relying on simulated datasets did not quantitatively evaluate the similarity to real data, which undermines their recommendations. Our simulation framework implants calibrated signals into real taxonomic profiles, including signals mimicking confounders. Using several whole meta-genome and 16S rRNA gene amplicon datasets, we validate that our simulated data resembles real data from disease association studies much more than in previous benchmarks. With extensively parametrized simulations, we benchmark the performance of nineteen differential abundance methods and further evaluate the best ones on confounded simulations. Only classic statistical methods (linear models, the Wilcoxon test, t-test), limma, and fastANCOM properly control false discoveries at relatively high sensitivity. When additionally considering confounders, these issues are exacerbated, but we find that adjusted differential abundance testing can effectively mitigate them. In a large cardiometabolic disease dataset, we showcase that failure to account for covariates such as medication causes spurious association in real-world applications. Tight error control is critical for microbiome association studies. The unsatisfactory performance of many differential abundance methods and the persistent danger of unchecked confounding suggest these contribute to a lack of reproducibility among such studies. We have open-sourced our simulation and benchmarking software to foster a much-needed consolidation of statistical methodology for microbiome research.

中文翻译：

---

人类微生物组研究中差异丰度测试和混杂因素调整的现实基准


在微生物组疾病关联研究中，测试哪些微生物在各组之间的丰度存在差异是一项基本任务。然而，对于差异丰度测试的合适或最佳统计方法缺乏共识，并且这些方法如何应对混杂因素仍有待探索。以前依赖模拟数据集的差异丰度基准没有定量评估与真实数据的相似性，这削弱了他们的建议。我们的模拟框架将校准信号植入真实的分类资料中，包括模仿混杂因素的信号。使用几个完整的元基因组和 16S rRNA 基因扩增子数据集，我们验证了我们的模拟数据比之前的基准更类似于疾病关联研究的真实数据。通过广泛的参数化模拟，我们对十九种差异丰度方法的性能进行了基准测试，并进一步评估了混杂模拟中的最佳方法。只有经典的统计方法（线性模型、Wilcoxon 检验、t 检验）、limma 和 fastANCOM 才能以相对较高的灵敏度正确控制错误发现。当另外考虑混杂因素时，这些问题会加剧，但我们发现调整后的差异丰度测试可以有效缓解这些问题。在大型心脏代谢疾病数据集中，我们展示了未能考虑药物等协变量会导致现实应用中的虚假关联。严格的误差控制对于微生物组关联研究至关重要。许多差异丰度方法的性能不令人满意，以及不受控制的混杂因素持续存在的危险，表明这些因素导致此类研究缺乏可重复性。 我们开源了我们的模拟和基准测试软件，以促进微生物组研究急需的统计方法的整合。