NISQA: Speech Quality and Naturalness Assessment
Overview
NISQA is a cutting-edge deep learning framework designed for assessing the quality and naturalness of speech. With the recent update to version 2.0, NISQA now offers multidimensional predictions with enhanced accuracy. This versatile tool allows users to train and fine-tune models for various applications, from phone call quality assessment to evaluating the naturalness of synthesized speech like those produced by Siri or Alexa.
Speech Quality Prediction
NISQA provides a comprehensive method for predicting speech quality. By utilizing deep learning model weights, it can evaluate the quality of speech that has passed through communication systems such as telephones or video calls. In addition to giving an overall quality score, NISQA's model also assesses specific dimensions of quality degradation which include:
- Noisiness: The level of background noise or interference.
- Coloration: Changes in the timbre or tone of the speech.
- Discontinuity: Interruptions or glitches in the audio stream.
- Loudness: Variations in the volume or intensity of the speech.
TTS Naturalness Prediction
For text-to-speech systems, NISQA offers the NISQA-TTS model weights, which are designed to estimate the naturalness of synthetic speech. This feature evaluates how "real" or human-like digitally generated speech sounds, which is crucial for virtual assistants and other speech-driven technologies.
Training and Fine-Tuning
Users have the flexibility to train new single-ended or double-ended speech quality models using different deep learning architectures like CNN or LSTM. The model weights provided by NISQA can be used to fine-tune these models for new data or transfer learning to different tasks such as emotion recognition or enhanced speech quality estimation.
Large Speech Quality Datasets
To support the development and refinement of speech quality models, NISQA offers a comprehensive dataset of over 14,000 speech samples. These samples come with subjective ratings of speech quality and are categorized into the aforementioned quality dimensions, making it an invaluable resource for research and practical applications.
Usage Guide
Installing NISQA
NISQA requires Anaconda for installation. The setup is straightforward, involving the creation of a dedicated environment to ensure all necessary dependencies are properly configured.
Performing Predictions
Users can predict speech quality through a command-line interface with options for single file, folder batch, or CSV-listed predictions. This allows for flexible integration into existing workflows or systems.
Training New Models
NISQA serves as a robust framework for training new models with customizable architectures. Users can opt for combinations like CNN-LSTM with various pooling strategies to suit their specific needs.
Evaluation
The trained models can be evaluated on given datasets, offering metrics such as Pearson's Correlation and RMSE. Optionally, it can produce detailed reports and diagrams to visualize model performance.
NISQA Corpus
The NISQA Corpus provides a rich array of more than 14,000 speech samples under various simulated and live conditions. This collection is essential for developing speech quality models as it includes diverse scenarios like codec use, packet loss, background noise, and more.
Licensing and Further Information
The NISQA framework and its components—models, datasets, and code—are distributed under various licenses, including MIT and Creative Commons. The project documentation and further details can be accessed through the NISQA paper and its Wiki, ensuring users have all the resources needed for their research or development projects.
For any usage of NISQA's models or corpus in academic research, appropriate citations to the original papers are required. The development team encourages scholarly engagement to enhance the field of speech quality and naturalness assessment.