NDR-code - Accurate 3D Reconstruction of Dynamic Scenes Using Monocular RGB-D Cameras

NDR-Code Project Overview

Introduction

The Neural-Dynamic Reconstruction (NDR) project introduces an innovative approach for capturing high-fidelity geometry, motion, and appearance of dynamic scenes using a monocular RGB-D camera. Unlike traditional methods, NDR is template-free, making it versatile for various applications in computer vision and graphics.

Key Features

High-Fidelity Capture: The project aims to recover detailed surface geometry and dynamic motion accurately, offering high-resolution reconstruction even in challenging environments.
Monocular Input: Utilizes a single RGB-D camera, making the setup simpler and more cost-effective compared to multi-camera systems.
Template-Free Approach: Eliminates the need for predefined templates, allowing for more flexibility in capturing diverse scenes and objects.

Data Structure

The data for the NDR project is organized similarly to the NeuS structure. This involves:

A directory for each case containing camera parameters (cameras_sphere.npz), depth maps, RGB images, and masks for each view.

Pre-Processed Data

NDR utilizes publicly available datasets, such as the KillingFusion dataset. Pre-processed data can be downloaded and used to kickstart experimentation, ensuring that licensing and citation guidelines are followed appropriately.

Getting Started

Setting up the NDR environment involves the following steps:

Clone the Repository:

git clone https://github.com/USTC3DV/NDR-code.git
cd NDR-code

Create the Environment:
```
conda env create -f environment.yml
conda activate ndr
```
Ensure that dependencies like PyTorch, OpenCV, and NumPy are correctly installed.

Running the Code

Training: Initiate the model training using:
```
python train_eval.py
```
Evaluation: Test pre-trained models with:
```
python pretrained_eval.py
```

Data Preparation

For implementing custom experiments, users need to prepare their data according to the specified conventions and perform pose initialization. Detailed steps are outlined in the project documentation.

Geometric Projection

To project reconstructed data:

Compile the necessary renderer:
```
cd renderer && bash build.sh && cd ..
```
Render meshes using:
```
python geo_render.py [path_to_data] [path_to_results] [iteration_number]
```
Output will be stored in the validations_geo directory in the specified results path.

Acknowledgements

The NDR project builds on foundational frameworks like NeuS and incorporates components from IDR and NeRF-pytorch. Contributions from other open-source projects like Fast-Robust-ICP and StereoPIFu_Code are acknowledged for their pivotal roles in camera pose initialization and geometry rendering.

Contact and Citation

For inquiries, you can reach out to Hongrui Cai through the provided contact channel. To cite this work in academic publications, use the provided BibTeX entry, ensuring proper attribution to the project's contributors.