Neural implicit representations, including Neural Distance Fields and Neural Radiance Fields, have demonstrated significant capabilities for reconstructing surfaces with complicated geometry and topology, and generating novel views of a scene. Nevertheless, it is challenging for users to directly deform or manipulate these implicit representations with large deformations in a real-time fashion. Gaussian Splatting (GS) has recently become a promising method with explicit geometry for representing static scenes and facilitating high-quality and real-time synthesis of novel views. However, it cannot be easily deformed due to the use of discrete Gaussians and the lack of explicit topology. To address this, we develop a novel GS-based method (GaussianMesh) that enables interactive deformation. Our key idea is to design an innovative mesh-based GS representation, which is integrated into Gaussian learning and manipulation. 3D Gaussians are defined over an explicit mesh, and they are bound with each other: the rendering of 3D Gaussians guides the mesh face split for adaptive refinement, and the mesh face split directs the splitting of 3D Gaussians. Moreover, the explicit mesh constraints help regularize the Gaussian distribution, suppressing poor-quality Gaussians ( e.g. , misaligned Gaussians, long-narrow shaped Gaussians), thus enhancing visual quality and reducing artifacts during deformation. Based on this representation, we further introduce a large-scale Gaussian deformation technique to enable deformable GS, which alters the parameters of 3D Gaussians according to the manipulation of the associated mesh. Our method benefits from existing mesh deformation datasets for more realistic data-driven Gaussian deformation. Extensive experiments show that our approach achieves high-quality reconstruction and effective deformation, while maintaining the promising rendering results at a high frame rate (65 FPS on average on a single commodity GPU).

中文翻译：

---

高斯飞溅的实时大规模变形


神经隐式表示，包括神经距离场和神经辐射场，已经展示了在重建具有复杂几何和拓扑的表面以及生成场景的新视图方面的重要能力。然而，用户很难直接以实时方式变形或操纵这些具有大变形的隐式表示。高斯展开 （GS） 最近已成为一种很有前途的方法，具有显式几何图形，用于表示静态场景并促进新视图的高质量和实时合成。然而，由于使用了离散的高斯分布和缺乏显式拓扑，它不容易变形。为了解决这个问题，我们开发了一种基于 GS 的新型方法 （GaussianMesh），可实现交互式变形。我们的关键思想是设计一种创新的基于网格的 GS 表示，并将其集成到高斯学习和操作中。3D 高斯分布在显式网格上定义，并且它们彼此绑定：3D 高斯的渲染指导网格面分割以进行自适应细化，而网格面分割则指导 3D 高斯分割。此外，显式网格约束有助于正则化高斯分布，抑制质量较差的高斯分布（例如，未对齐的高斯、长窄形状的高斯），从而提高视觉质量并减少变形过程中的伪影。基于这种表示，我们进一步引入了一种大规模的高斯变形技术来实现可变形 GS，它根据相关网格的操作改变 3D 高斯的参数。我们的方法受益于现有的网格变形数据集，可实现更真实的数据驱动高斯变形。 广泛的实验表明，我们的方法实现了高质量的重建和有效的变形，同时在高帧率（在单个商用 GPU 上平均为 65 FPS）保持了有希望的渲染结果。