Medical-SAM2

Overview of Medical SAM 2

Medical SAM 2, often abbreviated as MedSAM-2, is a cutting-edge model designed for the segmentation of medical images. It leverages the advanced SAM 2 framework to efficiently handle both 2D and 3D segmentation tasks in the medical field. The development of this model is detailed in a paper titled "Medical SAM 2: Segment Medical Images As Video Via Segment Anything Model 2." The paper provides in-depth insights into how MedSAM-2 applies techniques typically used for video processing to the segmentation of medical images, aiming to enhance accuracy and efficiency.

🔥 Key Features

Medical SAM 2 uniquely adapts the existing SAM 2 (Segment Anything Model 2) for medical applications, addressing the challenges of segmenting complex 3D medical images. It implements innovative methodologies that consider medical images similarly to how videos are processed, hence enhancing the capability to segment tissues, organs, and other structures in both two and three dimensions.

🩻 3D Abdomen Segmentation Visualization

One of the impressive capabilities of MedSAM-2 is its visualization of 3D abdomen segmentation. This feature provides a clear representation of how the model can segment multiple abdominal organs, offering substantial aid in medical diagnostics and planning.

🧐 System Requirements and Setup

To utilize MedSAM-2, it is essential to set up the environment correctly:

• Install the environment by executing the following commands:

  conda env create -f environment.yml
  conda activate medsam2
  

• Download the SAM2 checkpoint required for the process:

  bash download_ckpts.sh
  

This setup was tested on a system running Ubuntu 22.04, Conda version 23.7.4, and Python version 3.12.4. Users might need to troubleshoot based on their system variations.

🎯 Example Use Cases

2D Case - Optic-Cup Segmentation from Fundus Images

1. Dataset Preparation: Users should download the REFUGE dataset, which can be done either manually or via command lines.

   wget https://huggingface.co/datasets/jiayuanz3/REFUGE/resolve/main/REFUGE.zip
   unzip REFUGE.zip
   

2. Training & Validation: Execute the script to train and validate the model on the dataset.

   python train_2d.py -net sam2 -exp_name REFUGE_MedSAM2 -vis 1 -sam_ckpt ./checkpoints/sam2_hiera_small.pt -sam_config sam2_hiera_s -image_size 1024 -out_size 1024 -b 4 -val_freq 1 -dataset REFUGE -data_path ./data/REFUGE
   

3D Case - Abdominal Multiple Organs Segmentation

1. Dataset Preparation: Acquire the BTCV dataset similar to the 2D case.

   wget https://huggingface.co/datasets/jiayuanz3/btcv/resolve/main/btcv.zip
   unzip btcv.zip
   

2. Training & Validation: Run the corresponding script to train and validate for 3D segmentation.

   python train_3d.py -net sam2 -exp_name BTCV_MedSAM2 -sam_ckpt ./checkpoints/sam2_hiera_small.pt -sam_config sam2_hiera_s -image_size 1024 -val_freq 1 -prompt bbox -prompt_freq 2 -dataset btcv -data_path ./data/btcv
   

🚨 Latest Updates

• As of August 5, 2024, Medical SAM 2 paper ranked as the top paper of the day, indicating community interest and recognition.
• Details and links for dataset downloads and examples have been updated, ensuring users have access to the latest resources.
• The paper discussing Medical SAM 2 became publicly accessible, further illustrating the project's impact and reach in the academic community.

📝 Citation

For those interested in citing the work, you can reference the paper as follows:

@misc{zhu_medical_2024,
	title={Medical SAM 2: Segment medical images as video via Segment Anything Model 2},
    author={Jiayuan Zhu and Yunli Qi and Junde Wu},
    year = {2024},
    eprint={2408.00874},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

Medical SAM 2 represents a significant advancement in the field of medical imaging, providing innovative approaches to segment complex medical images effectively.