Reconstructing and rendering 3D objects from highly sparse views is of critical importance for promoting applications of 3D vision techniques and improving user experience. However, images from sparse views only contain very limited 3D information, leading to two significant challenges: 1) Difficulty in building multi-view consistency as images for matching are too few; 2) Partially omitted or highly compressed object information as view coverage is insufficient. To tackle these challenges, we propose GaussianObject, a framework to represent and render the 3D object with Gaussian splatting that achieves high rendering quality with only 4 input images. We first introduce techniques of visual hull and floater elimination, which explicitly inject structure priors into the initial optimization process to help build multi-view consistency, yielding a coarse 3D Gaussian representation. Then we construct a Gaussian repair model based on diffusion models to supplement the omitted object information, where Gaussians are further refined. We design a self-generating strategy to obtain image pairs for training the repair model. We further design a COLMAP-free variant, where pre-given accurate camera poses are not required, which achieves competitive quality and facilitates wider applications. GaussianObject is evaluated on several challenging datasets, including MipNeRF360, OmniObject3D, OpenIllumination, and our-collected unposed images, achieving superior performance from only four views and significantly outperforming previous SOTA methods.

中文翻译：

---

GaussianObject：使用 Gaussian Splatting 从四个视图进行高质量的 3D 对象重建


从高度稀疏的视图中重建和渲染 3D 对象对于促进 3D 视觉技术的应用和改善用户体验至关重要。然而，来自稀疏视图的图像仅包含非常有限的 3D 信息，这导致了两个重大挑战：1） 由于用于匹配的图像太少，难以构建多视图一致性;2） 由于视图覆盖率不足，部分遗漏或高度压缩对象信息。为了应对这些挑战，我们提出了 GaussianObject，这是一个使用 Gaussian 展开表示和渲染 3D 对象的框架，只需 4 个输入图像即可实现高渲染质量。我们首先介绍了视觉外壳和浮动消除技术，该技术将结构先验显式注入初始优化过程，以帮助构建多视图一致性，从而产生粗略的 3D 高斯表示。然后，我们在扩散模型的基础上构建一个高斯修复模型，以补充省略的对象信息，其中高斯被进一步细化。我们设计了一个自生成策略来获得用于训练修复模型的图像对。我们进一步设计了一种无 COLMAP 的变体，其中不需要预先给定的准确相机姿势，从而实现有竞争力的质量并促进更广泛的应用。GaussianObject 在几个具有挑战性的数据集上进行了评估，包括 MipNeRF360、OmniObject3D、OpenIllumination 和我们收集的未摆姿势图像，仅在四个视图中就实现了卓越的性能，并且明显优于以前的 SOTA 方法。