Introduction to GMFlow
The GMFlow project is a cutting-edge optical flow estimation method that innovatively redefines optical flow as a global matching task. The project features an official PyTorch implementation and was introduced in the paper titled "GMFlow: Learning Optical Flow via Global Matching," which was presented at the CVPR 2022 conference.
Key Features
GMFlow provides several standout features that make it an attractive tool for optical flow estimation:
-
Flexible and Modular Design: The project breaks down the optical flow framework into five main components: feature extraction, feature enhancement, feature matching, flow propagation, and flow refinement. This modular design allows users to easily customize their optical flow models by combining these different parts.
-
High Accuracy: GMFlow achieves impressive accuracy rates, even surpassing the RAFT model that requires 31 refinements, while GMFlow needs only one to outperform it on the challenging Sintel benchmark.
-
High Efficiency: The basic GMFlow model operates swiftly, taking just 57 milliseconds on a V100 GPU or 26 milliseconds on an A100 GPU for Sintel dataset computations. It bypasses the need for large volumes of sequential computations, especially on high-end GPUs, and simplifies backward flow computation. This efficiency also supports occlusion detection using a forward-backward consistency check mechanism with bidirectional flow.
Installation
The project is designed to be compatible with PyTorch 1.9.0, CUDA 10.2, and Python 3.8, though newer versions of PyTorch should work as well. Installation is facilitated using conda:
conda env create -f environment.yml
conda activate gmflow
Demonstrations
The GMFlow project provides pretrained models, which can be downloaded and utilized to run demonstrations on image sequences. Users can visualize results and predict bidirectional flow, with additional features such as forward-backward consistency checks for enhanced results.
Datasets
To train and evaluate GMFlow, several datasets are used:
- FlyingChairs
- FlyingThings3D
- Sintel
- KITTI
- HD1K
These datasets must be organized within a datasets directory. Alternatively, paths in the code can be adjusted if datasets are differently organized.
Evaluation
For evaluating trained GMFlow models, specific scripts are provided. Users can run the command to assess performance on datasets, with further scripts available for submissions to Sintel and KITTI online test sets.
Training
Training GMFlow involves detailed scripts for various datasets. The training setups vary, requiring different GPU configurations depending on whether the model includes refinement. The project supports monitoring and visualizing the training process via TensorBoard.
Acknowledgements
The success of GMFlow is built upon several outstanding repositories such as RAFT, LoFTR, DETR, Swin, mmdetection, and Detectron2. The contributions of these projects were crucial in developing GMFlow.
In essence, GMFlow is a powerful, flexible, and efficient approach to optical flow estimation, offering a substantial leap in both performance and usability for researchers and developers in the field.