gaustudio

GauStudio: Accelerating 3D Gaussian Splatting Research

GauStudio is an innovative modular framework designed to support and expedite research and development in the emerging field of 3D Gaussian Splatting (3DGS) and its various applications.

Dataset Overview

To ensure robust evaluation of 3DGS methodologies across different scenarios, GauStudio offers thorough datasets reflecting diverse lighting, materials, and geometric structures.

Synthetic Datasets in COLMAP Format

The framework includes five comprehensive synthetic datasets, namely nerf_synthetic, refnerf_synthetic, nero_synthetic, nsvf_synthetic, and BlendedMVS. These datasets cover 143 complex, real-world scenarios, all converted to COLMAP format for compatibility.

Real-world Scenes with Detailed Annotations

• MuSHRoom Data: Simplifies access to indoor scene data by converting the publicly available MuSHRoom dataset to COLMAP-compatible data.
• Tanks and Temples: Enhances this test set's evaluation by providing pose data in COLMAP format, expanding algorithm testing to more indoor and outdoor scenes.
• Advanced Annotations: Provides high-quality, temporally consistent data for complex scenarios, including sparse viewpoints and regions with specular highlights.

Installation Instructions

To get started with GauStudio, users need a setup that includes an NVIDIA graphics card with at least 6GB VRAM and a compatible CUDA installation. Python 3.8 or later is required.

Steps for Installation:

1. Create a Conda Environment:

   conda create -n gaustudio python=3.8
   conda activate gaustudio
   

2. Install PyTorch: PyTorch needs to be installed with versions such as torch1.12.1+cu113 or torch2.0.1+cu118.

3. Install Required Dependencies:

   pip install -r requirements.txt
   

4. Install Custom Rasterizer and GauStudio: Navigate to the proper directories and perform setup installations.

QuickStart Guide

The QuickStart guide facilitates users in extracting meshes from 3DGS data, offering support for most gaussian splatting methods. The structured input and output directories help in organizing data effectively for processing.

Mesh Extraction Instructions

1. Prepare the input directory with essentials like cameras.json and point cloud data structures.
2. Execute mesh extraction using gs-extract-mesh command, specifying input and output paths.

Future Plans and Contributions

GauStudio aims to expand its capabilities with future releases, including methods like Semi-Dense MVSplat-based and DepthAnything-based Gaussian Initializations, as well as full training pipelines and model enhancements like Gaussian Sky Modeling.

Citation and Contact

Researchers who find GauStudio beneficial are encouraged to cite the project in their work:

@article{ye2024gaustudio,
  title={GauStudio: A Modular Framework for 3D Gaussian Splatting and Beyond},
  author={Ye, Chongjie and Nie, Yinyu and Chang, Jiahao and Chen, Yuantao and Zhi, Yihao and Han, Xiaoguang},
  journal={arXiv preprint arXiv:2403.19632},
  year={2024}
}

For commercial collaborations or to address unresolved issues in 3DGS, contact Chongjie at [email protected]. GauStudio is released under the MIT License, making it available for both academic and commercial use.