NeuralNote: Transforming Audio into MIDI
NeuralNote is an innovative audio plugin that allows users to convert audio to MIDI effortlessly within their preferred Digital Audio Workstation (DAW). It offers a remarkable leap in music production by translating audio from any tonal instrument, including the human voice, into MIDI in real-time.
Key Features
-
Tonal Instrument Compatibility: NeuralNote efficiently transcribes audio from any tonal source including, but not limited to, string instruments, keyboards, and vocals.
-
Polyphonic Transcription: It supports the transcription of multiple notes played simultaneously, which is essential for polyphonic instruments like pianos and guitars.
-
Pitch Bend Detection: This feature captures the nuances of pitch variations, ensuring the MIDI output is as close as possible to the original performance.
-
Quick and Lightweight: Designed to be fast and lightweight, the plugin ensures seamless integration without overburdening the system resources.
-
Customizable Parameters: Users can adjust transcription parameters while listening to the conversion in real-time, providing flexibility and control over the output.
-
MIDI Scaling and Time Quantization: Once transcribed, users can fine-tune the MIDI output directly in the plugin, adjusting scales and timing to fit their musical needs perfectly.
Installation
NeuralNote supports both Windows and macOS. Users can download the latest version from GitHub. While installers are yet to be developed, the software can currently be installed by manually placing the plugin files in specific directories. Note that while the macOS version is signed, the Windows version might require additional steps for installation.
How to Use NeuralNote
NeuralNote can be utilized as an AudioFX plugin (available as VST3, AU, or standalone app) and is incorporated into tracks for transcription. Users can:
-
Load Audio: You can either record audio directly, function in tandem with your DAW's playback, or simply drag and drop audio files (.wav, .aiff, .flac, .mp3, .ogg) into the plugin.
-
Transcribe and Review: The MIDI transcription appears instantly in the piano roll section, where users can play the transcription back and tweak settings for optimal results.
-
Export MIDI: Once satisfied with the transcription, the MIDI file can be easily exported via drag and drop into a MIDI track.
Technical Insights
NeuralNote builds on Spotify's basic-pitch model and incorporates RTNeural for CNN processing, harnessing ONNXRuntime for feature processing. The plugin has made significant contributions, including adding 2D convolution support to RTNeural, enhancing compatibility and performance of the transcription engine.
Building from Source
For those interested in building NeuralNote from the source, the process requires git, cmake, and an appropriate compiler. Step-by-step instructions are provided for both macOS and Windows systems, with additional guidelines for handling specific Windows build constraints.
Future Enhancements
NeuralNote's roadmap includes improvements in stability, saving internal plugin states, enhancing MIDI handling for pitch bends, and expanding routing options for MIDI output.
Community and Contribution
The NeuralNote project is open-source and licensed under Apache-2.0. Developers and users are encouraged to contribute to the project by improving features or documentation. Bug reports and feature suggestions are welcomed through GitHub issues.
Credits and Thanks
Created by Damien Ronssin and Tibor Vass, with UI design by Perrine Morel, NeuralNote acknowledges its contributors for their enhancements, including improvements in scaler options and file browsing interfaces.
NeuralNote stands as a testament to the collaborative spirit of open-source development and continues to evolve with contributions and feedback from its user community.