resemble-enhance - AI Technology for Speech Denoising and Enhancement

Resemble Enhance Project Introduction

Resemble Enhance is a cutting-edge AI-powered tool designed to significantly improve speech quality by performing tasks such as denoising and enhancement. Developed to streamline audio clarity, it is composed of two primary modules: a denoiser and an enhancer.

Features of Resemble Enhance

Denoiser: This module functions by isolating speech from its noisy background, ensuring that the speech captured is as clear as possible.
Enhancer: Following the denoising process, this module improves the perceptual audio quality. It achieves this by restoring any lost audio components and extending the bandwidth of the audio signal to deliver vibrant, high-quality speech.

Both modules are trained using high-quality 44.1kHz speech data to ensure that the speech is enhanced with the highest possible fidelity.

How to Use Resemble Enhance

Installation

To get started with Resemble Enhance, you can easily install it via pip. For the stable version, you can run:

pip install resemble-enhance --upgrade

For those interested in the latest features, the pre-release version can be installed with:

pip install resemble-enhance --upgrade --pre

Execution

To Enhance: Transform your audio files by running the following command:
```
resemble_enhance in_dir out_dir
```
To Denoise Only: If your focus is solely on denoising, utilize this command:
```
resemble_enhance in_dir out_dir --denoise_only
```

Web Demo

To experience the capabilities of Resemble Enhance, a web demo is available. Built with Gradio, users can try it out online through the Hugging Face Space. Additionally, users can run this demo locally using Python:

python app.py

Training Your Own Model

For those interested in customizing or experimenting with their models, Resemble Enhance offers comprehensive guidelines on preparing datasets and training models.

Data Preparation

Prepare datasets including foreground speech, background non-speech, and room impulse responses (RIR).

data
├── fg
│   ├── 00001.wav
│   └── ...
├── bg
│   ├── 00001.wav
│   └── ...
└── rir
    ├── 00001.npy
    └── ...

Training Steps

Denoiser Warmup: Although the denoiser is trained jointly with the enhancer, it's advisable to first conduct a warmup training session:
```
python -m resemble_enhance.denoiser.train --yaml config/denoiser.yaml runs/denoiser
```

Enhancer Training: This is accomplished in two stages:

Stage 1: First, train the autoencoder and vocoder:

python -m resemble_enhance.enhancer.train --yaml config/enhancer_stage1.yaml runs/enhancer_stage1

Stage 2: Next, focus on training the latent conditional flow matching (CFM) model:

python -m resemble_enhance.enhancer.train --yaml config/enhancer_stage2.yaml runs/enhancer_stage2

Conclusion

For those eager to delve deeper, comprehensive information and updates about Resemble Enhance are regularly published on their official website. Resemble Enhance stands as a testament to modern technological advancements in artificial intelligence and audio processing, helping elevate the standard of audio clarity and quality in various applications.