Introduction to EfficientSAM
EfficientSAM stands for "Leveraged Masked Image Pretraining for Efficient Segment Anything." This innovative project focuses on advanced image segmentation techniques, which are crucial for various applications in computer vision. By leveraging pre-trained masked images, EfficientSAM aims to efficiently adapt to different segmentation tasks and settings.
Recent Developments
EfficientSAM has seen several important updates and developments:
- January 12, 2024: The ONNX version of EfficientSAM, including separate encoder and decoder components, is now available on Hugging Face Space. This move was facilitated by Kentaro Wada.
- December 31, 2023: Integration with the annotation tool, Labelme, has been successfully completed, enhancing annotation capabilities.
- December 11, 2023: The complete EfficientSAM model code, including checkpoints, is now accessible within the project repository. An example script is provided to demonstrate model usage.
- December 10, 2023: A demo version of Grounded EfficientSAM is now available, courtesy of the IDEA-Research team.
- December 6, 2023: An EfficientSAM demo was made available on Hugging Face Space.
- December 5, 2023: The release of the torchscript version of EfficientSAM was announced, alongside a colab for practical use.
Online Demo and Examples
For those who wish to explore EfficientSAM in action, an online demo along with examples is provided on the project page.
Segmentation Examples
EfficientSAM demonstrates versatile segmentation capabilities, evident through various examples:
- Point-prompt: An interactive method allowing users to specify points to guide segmentation.
- Box-prompt: Users can define bounding boxes to enhance segmentation outcomes.
- Segment everything: This approach attempts to segment all discernible objects within an image.
- Saliency: Emphasizes the most prominent elements within the scene.
Model Information
EfficientSAM offers different model checkpoints located in the repository's weights folder. Users can experiment with these models using the EfficientSAM_example.py.
- EfficientSAM-S and EfficientSAM-Ti can be downloaded for practical implementation. Here’s a snippet illustrating basic usage:
from efficient_sam.build_efficient_sam import build_efficient_sam_vitt, build_efficient_sam_vits
efficientsam = build_efficient_sam_vitt()
Jupyter Notebook Example
An example in Jupyter Notebook is also available, which provides a more interactive method to explore model functionalities. The notebook is shared here.
Acknowledgements
Development of EfficientSAM is built upon several notable projects, including SAM, MobileSAM, FastSAM, and U-2-Net.
Citation
Researchers and developers using EfficientSAM in their work are encouraged to cite it via the following BibTeX entry:
@article{xiong2023efficientsam,
title={EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything},
author={Yunyang Xiong, Bala Varadarajan, Lemeng Wu, Xiaoyu Xiang, Fanyi Xiao, Chenchen Zhu, Xiaoliang Dai, Dilin Wang, Fei Sun, Forrest Iandola, Raghuraman Krishnamoorthi, Vikas Chandra},
journal={arXiv:2312.00863},
year={2023}
}
EfficientSAM provides a comprehensive toolset for efficient image segmentation, facilitating enhanced performance and versatility across various applications.