Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes
Overview
Gaussian Opacity Fields (GOF) is an innovative approach for geometry extraction using 3D Gaussians. This method highlights the ability to directly identify a 3D shape's level set, enabling precise surface reconstruction. A significant aspect of GOF is its regularization strategy, which enhances the reconstruction quality. Additionally, the project employs Marching Tetrahedra techniques to ensure adaptive and compact mesh extraction, making it ideal for vast, unbounded scenes.
Recent Updates
- September 11, 2024: GOF has gained acceptance at SIGGRAPH ASIA 2024, emphasizing its importance in the field of computer graphics. The research paper associated with this project has been updated to include more comprehensive details, explanations, and analysis.
- June 10, 2024: The project achieved a major milestone by doubling its training speed using merged operations. Now, six scenes from the TNT dataset can be trained in approximately 24 minutes, while the bicycle scene from the Mip-NeRF 360 dataset takes around 45 minutes.
Installation Guide
To get started with Gaussian Opacity Fields, follow these installation steps:
- Clone the repository using Git.
- Set up a new conda environment with Python version 3.8.
- Install the necessary packages and tools, such as PyTorch and CUDA, using pip and conda.
- Complete the setup by installing all required submodules and configuring the environment for triangulation and mesh extraction.
Datasets
The project leverages several datasets to evaluate and demonstrate the effectiveness of its methods:
- Mip-NeRF 360 and NeRF-Synthetic datasets provide diverse scenes for testing.
- DTU and Tanks and Temples datasets offer ground truth point clouds for evaluating geometry reconstruction accuracy.
To utilize these datasets, download them from their respective sources and prepare them according to the specified directory structure.
Training and Evaluation
GOF supports multiple dataset formats and can be trained using specific scripts tailored to each dataset such as NeRF-synthetic, Mip-NeRF 360, Tanks and Temples, and DTU. For a custom dataset, maintain the 3DGS data format, and follow the provided instructions to train models and extract meshes.
Acknowledgements
GOF builds upon and integrates several other projects and tools to achieve its goals:
- 3DGS and Mip-Splatting provide foundational techniques and visualizations.
- Tetra-NeRF and Kaolin libraries contribute to the tetrahedra triangulation and marching tetrahedra processes respectively.
- Evaluation scripts originated from DTUeval-python and TanksAndTemples are also utilized.
The project extends gratitude to these contributors for their work and resources.
Citation
Researchers and developers who find the GOF project or its regularization methods useful are encouraged to cite the relevant articles, ensuring proper acknowledgment in academic and professional works.