Introduction to the Riffusion App
Riffusion is a fascinating app designed for those interested in music and technology. It offers real-time music generation, utilizing the advanced capabilities of stable diffusion. Although this project is no longer actively maintained, it provides a unique combination of web technologies to create an interactive space for music enthusiasts.
Technical Foundations
The Riffusion App is built on various modern technologies. At its heart lies Next.js, a popular framework for building server-side rendering and static web applications using React. Accompanying React, the app also incorporates Typescript for enhanced development efficiency, three.js for 3D animation and visualization, Tailwind for styling, and is hosted using Vercel, which streamlines deployment.
Getting Started
If you are keen to explore or modify the Riffusion App, you’ll need to have Node.js version 18 or newer installed. You can check your current version using the command node --version. Once you have the correct version, you’ll need to install the necessary packages using npm:
npm install
After setting up the environment, you can run the app in development mode by executing:
npm run dev
# or
yarn dev
Once running, navigate to http://localhost:3000 in your web browser to interact with the app. The main page is located in the pages/index.js file, and it conveniently updates in real-time as you make edits. For information about the app, you can look at the pages/about.tsx file.
The directory pages/api is specially configured to handle API routes, which is useful for backend requests and functionality.
Inference Server
A critical component of generating real-time music is the Inference Server. It acts as the backend, processing requests to produce musical outputs. Anyone looking to generate these model outputs will need a robust GPU capable of handling stable diffusion quickly, ideally in under five seconds.
To connect the web app with the inference server, you must clone and run the Flask app from a separate repository. Once done, create a .env.local file at the root of the Riffusion App repository. This file should include the URL to your locally running inference server, like so:
RIFFUSION_FLASK_URL=http://127.0.0.1:3013/run_inference/
Citing Riffusion
For those who wish to expand upon this work, acknowledging the creators is essential. When citing Riffusion, use the reference below:
@article{Forsgren_Martiros_2022,
author = {Forsgren, Seth* and Martiros, Hayk*},
title = {{Riffusion - Stable diffusion for real-time music generation}},
url = {https://riffusion.com/about},
year = {2022}
}
With its combination of innovative technology and music generation, Riffusion stands as an intriguing project for developers and music fans alike. While the project itself might no longer be under active maintenance, its groundwork continues to inspire exploration in real-time music and computational creativity.