Mip-Splatting: Alias-free 3D Gaussian Splatting
Mip-Splatting is a cutting-edge project in the realm of 3D graphics and visualization, designed by a team of talented researchers, namely Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. This project has garnered attention for its innovative approach presented at CVPR 2024, earning the Best Student Paper award.
Core Contribution
The core achievement of Mip-Splatting is its development of a 3D smoothing filter combined with a 2D Mip filter, specifically applied to 3D Gaussian Splatting (3DGS). This technology addresses and eliminates multiple visual artifacts, producing alias-free renderings, which is a significant leap forward in rendering quality.
Enhanced Features
In addition to its core contributions, Mip-Splatting builds upon the densification metric proposed in the Gaussian Opacity Fields project. This improvement significantly enhances the results of synthesizing novel views, making it a powerful tool for those working in 3D scene reconstruction and rendering.
Installation Guide
For those interested in exploring this technology, Mip-Splatting's code is available for public use. The installation process involves cloning the repository from GitHub and setting up an environment using Anaconda. The specific steps include:
- Cloning the repository:
git clone [email protected]:autonomousvision/mip-splatting.git cd mip-splatting - Creating and activating an Anaconda environment:
conda create -y -n mip-splatting python=3.8 conda activate mip-splatting - Installing necessary packages:
pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 -f https://download.pytorch.org/whl/torch_stable.html conda install cudatoolkit-dev=11.3 -c conda-forge pip install -r requirements.txt pip install submodules/diff-gaussian-rasterization pip install submodules/simple-knn/
Dataset Preparation
Mip-Splatting utilizes two primary datasets: the Blender Dataset and the Mip-NeRF 360 Dataset.
- Blender Dataset: Users need to download the
nerf_synthetic.zipfile from NeRF's official Google Drive. Once downloaded, a multi-scale Blender dataset can be generated. - Mip-NeRF 360 Dataset: The required data can be accessed from the Mip-NeRF 360 webpage. Additional scenes such as treehill and flowers are available upon request from the authors.
Training and Evaluation
With the datasets in place, training and evaluation can proceed with various scripts provided in the repository:
- Single-scale training and multi-scale testing on NeRF-synthetic dataset.
- Multi-scale training and testing on the same dataset.
- Options are available for both single-scale and multi-scale setups on the mip-nerf 360 dataset.
Visualization
After the training phase, Mip-Splatting provides tools to visualize the results. A 3D smoothing filter can be fused with Gaussian parameters to create a visual representation, viewable through the project's online viewer.
python create_fused_ply.py -m {model_dir}/{scene} --output_ply fused/{scene}_fused.ply"
Acknowledgements
Mip-Splatting is built upon the foundational work of the 3DGS project, reflecting collaborative and appreciative acknowledgment of prior research and contributions.
Citation
For those using Mip-Splatting in their work, the project documentation encourages citation of its formal papers presented at the CVPR conference and in related publications.
Mip-Splatting represents a frontier in the evolution of 3D rendering, poised to advance the capabilities in creating high-quality, realistic visual representations in various applications and industries.