Trajectory inference methods are essential for analyzing the developmental paths of cells in single-cell sequencing datasets. It provides insights into cellular differentiation, transitions, and lineage hierarchies, helping unravel the dynamic processes underlying development and disease progression. However, many existing tools lack a coherent statistical model and reliable uncertainty quantification, limiting their utility and robustness. In this paper, we introduce VITAE (Variational Inference for Trajectory by AutoEncoder), a statistical approach that integrates a latent hierarchical mixture model with variational autoencoders to infer trajectories. The statistical hierarchical model enhances the interpretability of our framework, while the posterior approximations generated by our variational autoencoder ensure computational efficiency and provide uncertainty quantification of cell projections along trajectories. Specifically, VITAE enables simultaneous trajectory inference and data integration, improving the accuracy of learning a joint trajectory structure in the presence of biological and technical heterogeneity across datasets. We show that VITAE outperforms other state-of-the-art trajectory inference methods on both real and synthetic data under various trajectory topologies. Furthermore, we apply VITAE to jointly analyze three distinct single-cell RNA sequencing datasets of the mouse neocortex, unveiling comprehensive developmental lineages of projection neurons. VITAE effectively reduces batch effects within and across datasets and uncovers finer structures that might be overlooked in individual datasets. Additionally, we showcase VITAE’s efficacy in integrative analyses of multiomic datasets with continuous cell population structures.

中文翻译：

---

使用深度学习和混合先验进行单细胞基因组学的联合轨迹推断


轨迹推断方法对于分析单细胞测序数据集中细胞的发育路径至关重要。它提供了对细胞分化、转变和谱系层次结构的见解，有助于揭示发育和疾病进展的动态过程。然而，许多现有工具缺乏连贯的统计模型和可靠的不确定性量化，限制了它们的实用性和鲁棒性。在本文中，我们介绍了 VITAE（自动编码器的轨迹变分推理），这是一种将潜在分层混合模型与变分自动编码器集成来推断轨迹的统计方法。统计分层模型增强了我们框架的可解释性，而我们的变分自动编码器生成的后验近似确保了计算效率并提供沿轨迹的细胞投影的不确定性量化。具体来说，VITAE 能够同时进行轨迹推断和数据集成，从而在跨数据集存在生物和技术异质性的情况下提高学习联合轨迹结构的准确性。我们证明，VITAE 在各种轨迹拓扑下的真实数据和合成数据上都优于其他最先进的轨迹推理方法。此外，我们应用 VITAE 联合分析小鼠新皮质的三个不同的单细胞 RNA 测序数据集，揭示投射神经元的全面发育谱系。 VITAE 有效地减少了数据集内部和数据集之间的批次效应，并发现了单个数据集中可能被忽视的更精细的结构。此外，我们还展示了 VITAE 在对具有连续细胞群结构的多组学数据集进行综合分析方面的功效。