Project Introduction to Scenic
Scenic is a dynamic codebase designed to propel research in attention-based models specifically tailored for computer vision. It has proven effective in developing various models focused on classification, segmentation, and detection across multiple modalities, including images, video, audio, and combinations of these.
What is Scenic?
At its core, Scenic functions as a collection of lightweight, shared libraries geared towards tackling common tasks encountered during the training of large-scale vision models. Additionally, it showcases specific projects that incorporate comprehensive training and evaluation processes, leveraging these libraries. Scenic is developed using JAX and relies on Flax.
Key Features
Scenic offers a range of features, including:
- Essential code templates for launching experiments, summary writing, logging, and profiling.
- Optimized training and evaluation loops, metrics, and more.
- Input pipelines tailored for popular vision datasets.
- Access to robust baseline models, including non-attentional alternatives.
Cutting-edge Models and Baselines
Scenic has been employed in the development of state-of-the-art (SOTA) models and baselines. Projects utilizing or developed within Scenic have expanded its capabilities significantly.
Notable projects include:
- ViViT for video vision transformation
- OmniNet for omnidirectional representations
- Further exploration of pre-training limits and multi-modal fusion techniques
Additionally, Scenic has reproduced various baseline models, such as:
- Vision Transformers (ViT)
- DEformable TRANSformers (DETR)
- Several other advanced models, demonstrating the versatility of Scenic.
Philosophy and Design
Scenic aims to streamline the prototyping of large-scale vision models by maintaining simplicity in code. It emphasizes direct modifications, preferring straightforward forking and copy-pasting over complex abstractions, ensuring ease of understanding and extension.
Getting Started
To get started with Scenic:
- Ensure you have Python 3.9 or newer installed.
- Clone the Scenic repository from GitHub.
- Navigate into the directory and install necessary dependencies.
- Run training processes, such as ViT on ImageNet, with designated configuration files.
For specific projects, additional packages may be required as noted in their README or requirements files.
Component Design
Scenic’s architecture promotes flexibility by offering:
- Library-level Code: Minimal and well-tested, shared libraries for data pipelines, model interfaces, etc.
- Project-level Code: Customized solutions for unique tasks and data scenarios. It can involve merely altering hyperparameters or entirely redefining models, metrics, and more.
Community and Contribution
Those interested in contributing to Scenic can explore the philosophy, code structure, and contributing guidelines. Contributions that enhance shared libraries are welcomed through pull request submissions.
Citations
For those utilizing Scenic in their research, a formal citation can be made using its white paper, available in academic repositories.
Scenic stands as a comprehensive toolkit, fostering innovation and exploration in computer vision through its adaptable and robust framework. By offering a blend of foundational libraries and project-specific tools, it caters to the varied needs of the research community in advancing model development and experimentation.