CPM: Color-Pattern Makeup Transfer
The CPM (Color-Pattern Makeup Transfer) is an innovative framework developed for transferring makeup styles from one image to another, exceeding previous models in performance across various makeup styles, including both subtle and bold looks. This framework introduces novel methods for transferring not just the color but also the intricate patterns of makeup. Here's a closer look at what CPM offers:
Datasets
CPM introduces four new datasets specifically designed for training and evaluating this makeup transfer model. These datasets include both real-world and synthetic images:
- CPM-Real: Comprising 3,895 images featuring real makeup styles, this dataset offers a variety of authentic makeup looks.
- CPM-Synt-1: Consists of 5,555 synthetic images, which include pattern segmentation masks, facilitating detailed analysis and model training.
- CPM-Synt-2: A set of 1,625 synthetic triplets providing images with and without makeup, along with corresponding ground truth, to aid model comparison.
- Stickers: Includes 577 high-quality images with an alpha channel, useful for overlay and pattern analysis.
These datasets are available for research purposes and offer extensive data for enhancing makeup transfer techniques.
Getting Started
Setting up CPM is straightforward with the following prerequisites and steps:
Requirements
- Python version 3.7
- PyTorch version 1.6.0
- TensorFlow-GPU version 1.14
- Segmentation models PyTorch library
Installation
You can start by cloning the CPM repository and setting up the necessary environment:
# Cloning the repository
git clone https://github.com/VinAIResearch/CPM.git
cd CPM
# Installing dependencies via Conda
conda env create -f environment.yml
Pre-trained Models
Download the pre-trained models required for color and pattern transfer:
mkdir checkpoints
cd checkpoints
wget https://public.vinai.io/CPM_checkpoints/color.pth
wget https://public.vinai.io/CPM_checkpoints/pattern.pth
You can also download the PRNet pre-trained model separately.
Usage Example
Once everything is set up, you can run CPM in three modes: transferring both color and patterns, only color, or only patterns. Here’s an example command for transferring both:
CUDA_VISIBLE_DEVICES=0 python main.py --style ./imgs/style-1.png --input ./imgs/non-makeup.png
The resulting image will be saved in the specified directory. This process allows for various makeup styles to be virtually applied with remarkable accuracy.
Training and Evaluation
CPM's modular design comprises a distinct workflow for color and pattern branches. While they are independent, both branches follow similar steps:
- Data Preparation: This involves generating texture maps of faces to facilitate style transfer.
- Training: Detailed training steps are documented for both color and pattern branches, guiding users through the process of training their models using available datasets.
Troubleshooting tips are provided to resolve common issues during setup or execution, such as addressing library import errors or CUDA device compatibility issues.
CPM not only provides advanced tools for makeup transfer but also paves the way for future research in the field of computer vision and appearance transformation. Researchers and developers are encouraged to utilize the datasets and contribute to ongoing improvements and adaptability of the framework.