(Generic) EfficientNets for PyTorch: An Overview
EfficientNets for PyTorch is a versatile library designed for utilizing compute and parameter-efficient network architectures. Although the repository is no longer actively maintained, it offers a valuable reference to models such as EfficientNet, MixNet, and MobileNetV3, which are highly efficient in neural architecture search.
Project Purpose
The project provides implementation templates for a variety of lightweight models, focusing on string-based architecture definitions. These models rely on the MobileNet V1/V2 block sequence principles, allowing users to easily replicate or adapt cutting-edge neural architectures discovered through automated search methods.
Key Features and Updates
Notable Recent Improvements
- August 19, 2020: Addition of new EfficientNet-B3 weights trained with PyTorch, featuring a top-1 accuracy of 82.1%. The repository also gained EfficientNet-Lite0 weights for improved performance.
- April 5, 2020: Introduction of MobileNet-V2 models demonstrating competitive performance, such as the MobileNet-V2 100, which requires only 3.5 million parameters yet achieves 73% top-1 accuracy.
- March 23, 2020: Release of EfficientNet-Lite models and various MobileNet-V3 model weights, together with important weight initialization updates.
- February 12, 2020: Addition of EfficientNet-L2 and B0-B7 models featuring "NoisyStudent" weights, known for robust learning performance. The implementation also expanded to include RandAugment-trained EfficientNet-ES weights.
Model Variants
The library encompasses numerous model architectures, ported from official TensorFlow repositories where applicable:
- EfficientNet NoisyStudent (B0-B7, L2): Renowned for its strong performance metrics on large datasets.
- EfficientNet AdvProp (B0-B8): Enhanced models utilizing adversarial propagation.
- EfficientNet-EdgeTPU, CondConv, Lite: Models tailored for special-purpose hardware and lightweight settings.
- MixNet and MNASNet variants: Offer alternative approaches to network architecture efficiency.
- MobileNet-V3 and FBNet-C: Popular choices for mobile and embedded applications.
- Single-Path NAS: Models defined through the single-path neural architecture search paradigm.
Pretrained Models
The repository features a wide array of pretrained models capable of achieving competitive accuracy, closely matching or even surpassing original paper benchmarks:
- Models like EfficientNet-B3, MixNet-XL, and EfficientNet-Lite0 come with detailed accuracy statistics and parameter counts, assisting users in selecting the most suitable model for their needs.
Technical Insights and Tools
EfficientNets for PyTorch provides various technical tools and features, including:
- ONNX and Caffe2 export capabilities to support broader deployment needs.
- Compatibility with PyTorch scripting and optimized activation functions for improved model efficiency.
- Pre-integration of TensorFlow preprocessing mechanisms, enabling seamless model validation practices that mimic native TensorFlow conditions.
Usage Recommendations
While using this repository, it is recommended to transition to the timm library for continued updates and broader model coverage. The timm library encompasses all models found here and expands the available options considerably.
Despite being archived, (Generic) EfficientNets for PyTorch offers valuable insights into model efficiency and remains an important reference point for developers exploring lightweight model architectures.