AMC: AutoML for Model Compression
AutoML for Model Compression (AMC) is an innovative project designed to enhance the efficiency and performance of machine learning models on mobile devices. This project is grounded in research presented in the paper "AMC: AutoML for Model Compression and Acceleration on Mobile Devices" and offers solutions for compressing machine learning models efficiently while retaining their performance.
Overview
AMC leverages automated machine learning techniques to compress and accelerate models, making it particularly useful for mobile device applications where computing resources are limited. By optimizing the models' architecture, AMC improves speed and reduces computational demands without compromising accuracy.
Training AMC
The current implementation of AMC supports the automated pruning of the MobileNet model on the ImageNet dataset. This pruning process involves three main steps:
-
Strategy Search
- The first step is to find a suitable compression strategy for the MobileNet model. Users begin by downloading the pre-trained MobileNet, and then run a specific script to search for the compression strategy under a constraint, such as 50% FLOPs (Floating Point Operations per second).
-
Export the Pruned Weights
- After finding a suitable strategy, the next step is to export the pruned model weights. This process involves ensuring that the MobileNet file is adjusted to accommodate the pruned model.
-
Fine-Tuning from Pruned Weights
- Finally, the pruned model needs to be further refined or fine-tuned. This can be achieved by using a specific learning rate schedule over multiple epochs.
AMC Compressed Models
The AMC project provides models and their corresponding weights that have been compressed using its methods. Two variants of MobileNet—MobileNet-V1 and MobileNet-V2—are available in both PyTorch and TensorFlow formats. These models illustrate significant improvements in accuracy even with reduced FLOPs or execution time:
- MobileNetV1-50%FLOPs: Achieves 70.494% Top-1 accuracy and 89.306% Top-5 accuracy.
- MobileNetV2-70%FLOPs: Manages to reach 70.854% Top-1 accuracy and 89.914% Top-5 accuracy.
Technical Details and Requirements
To run the AMC project, certain dependencies are required. The codebase has been tested under the following environment:
- Python 3.7.3
- PyTorch 1.1.0
- torchvision 0.2.1
- NumPy 1.14.3
- SciPy 1.1.0
- scikit-learn 0.19.1
- TensorboardX for visualization
- ImageNet dataset for model training and testing
Contact Information
Inquiries about the project can be directed to the authors, Ji Lin at [email protected] and Song Han at [email protected]. They are approachable for questions and collaborations related to the project or its implementations.
In summary, AMC presents a powerful tool for compressing machine learning models, making them more suited for mobile application environments while retaining performance, thereby broadening the scope and usability of advanced AI technologies.