#Singing Voice Synthesis

DiffSinger employs a shallow diffusion mechanism for high-quality singing voice synthesis, providing tools for Text-to-Speech and Singing-Voice Synthesis. The project includes features like MIDI integration, pitch extraction, and advanced vocoders, including HiFiGAN and NSF-HiFiGAN. Designed for flexibility and speed, it incorporates enhancements like PNDM for faster processing and integrates seamlessly with datasets such as Ljspeech and OpenCpop. It includes comprehensive documentation and interactive demos on Hugging Face, with continuous development ensuring compatibility with PyTorch and features acknowledged in conferences like ACL and NeurIPS.

DiffSinger provides a PyTorch-based implementation for singing voice synthesis, using a shallow diffusion mechanism and pre-trained FastSpeech2 Auxiliary Decoder. It supports multiple model configurations—naive, auxiliary, and shallow—with TTS features like pitch, volume, and rate control. Designed for single and batch inference with pretrained models, the repository offers detailed guidance for comprehensive synthesis and training, while still in development for multi-speaker training.

The tool enables seamless conversion of MIDI files into singing voices using the HMM-based system from sinsy.jp developed by Nagoya Institute of Technology. It supports various languages and voice configurations, catering to lyrics, gender, tempo, and additional parameters. Requiring only Python 3 and MuseScore for setup, the project offers language support in Japanese, English, and Mandarin, with features testable via Colab. Created by Mathias Gatti, it is open-source, licensed under MIT, with ko-fi contributions appreciated.