MockingBird

MockingBird Project Introduction

Overview

MockingBird is an innovative project aimed at advancing voice cloning technology. It supports Mandarin Chinese and is built with the popular PyTorch framework. Despite the main repository no longer receiving updates, the project continues to grow with support from an open-source community.

Key Features

• Multilingual Support: The project supports Mandarin Chinese and has been tested with diverse datasets such as aidatatang_200zh, magicdata, and aishell3.

• Cross-Platform Compatibility: MockingBird runs smoothly on both Windows and Linux operating systems and is compatible with M1 MacOS.

• Effortless Integration: Users benefit from fast and effective results using a newly-trained synthesizer by leveraging the pretrained encoder/vocoder.

• Web-Ready: The system can serve results via remote requests, making it suitable for various applications.

System Requirements

Before diving into MockingBird, ensure your environment meets the following requirements:

1. Python Version: You need Python 3.7 or higher to run the toolbox.

2. Installations:

   • PyTorch: Obtain it from the official site based on your system.
   • ffmpeg: Essential for processing multimedia files.
   • Other required packages can be installed using pip install -r requirements.txt.

Quick Setup

General Setup

• Create a virtual environment and activate it using Conda or Mamba.
• Follow the instructions in the env.yml file to ensure all dependencies are installed.

M1 Mac Setup

For M1 Mac users, additional steps are needed due to compatibility issues, especially with PyQt5:

• Use a Rosetta Terminal and create a virtual environment.
• Install PyQt5, pyworld, and ctc-segmentation following the specialized guidelines for M1 processors.

Model Preparation

MockingBird allows you to use pre-trained models or train your own:

• Encoder Training: Preprocess audio data and train the encoder using supported datasets like librispeech_other and voxceleb1/2.

• Synthesizer Training: Download relevant datasets, preprocess them and train the synthesizer. This step is crucial because the original synthesize models are not compatible with Chinese symbols.

Additional Resources

For those interested, a demo video showcases the project's capabilities in action. This project adheres to the MIT license, emphasizing its open-source nature.

Conclusion

MockingBird is paving the way for advanced voice cloning applications by combining robust technology with community-supported development. Whether you're a developer or a tech enthusiast, this project offers a compelling entry point into the world of voice cloning.