flowty-realtime-lcm-canvas

Overview of flowty-realtime-lcm-canvas

The flowty-realtime-lcm-canvas project is an engaging and innovative demonstration that facilitates real-time sketch-to-image conversion. This project leverages LCM (Latent Contrastive Model) in combination with the Gradio library to bring this captivating experience to life. Not familiar with LCM? You can explore it in detail through this Hugging Face article.

With the help of LCM LoRA, flowty-realtime-lcm-canvas allows users to switch between different models effortlessly by modifying the model ID within the interface. The core aim of this project is to offer users a seamless experience where they can draw on one side of the interface and witness their sketches transform into images almost instantaneously on the other side.

It's worth noting that the performance of this real-time setup can vary significantly depending on the GPU in use. For instance, NVIDIA's 4090 GPUs tend to deliver the best performance in real-time scenarios. Users are encouraged to share their results and experiences!

System Performance and Testing

The project has been tested on a MacBook Pro equipped with an M2 Max chip featuring a 30-core GPU and 32GB of memory, running on Python 3.10. The inference time reported during testing was approximately 1.2 seconds per image render, which is deemed tolerable for real-time applications. Users achieving good performance on their machines can fine-tune several parameters to enhance results further. Additionally, adjustments to the canvas size (ranging from 768 to 1024) in the ui.py file can be made to suit different models.

How to Set Up

To get started with flowty-realtime-lcm-canvas, follow these setup instructions:

1. Setting Up a Virtual Environment:

   • Create a virtual environment to isolate your project's dependencies:

     python -m venv env
     

   • Activate the environment:
     • On macOS:

       source ./env/bin/activate
       

     • On Windows:

       env\Scripts\activate
       

2. Installing PyTorch for Nvidia Users:

   • Use the following command to install PyTorch:

     pip install torch --extra-index-url https://download.pytorch.org/whl/cu121
     

3. Installing Project Requirements:

   • Install all necessary dependencies by running:

     pip install -r requirements.txt
     

4. Running the Project:

   • Launch the user interface using the command:

     python ui.py
     

   • Models will be automatically downloaded to the models directory. The download duration will depend on your internet speed. Once ready, Gradio will display a URL in the console, which you can use to access the user interface.

Usage on Google Colab

Google Colab users can also access this project with these commands (note that it's typically confined to Colab Pro accounts):

!git clone https://github.com/flowtyone/flowty-realtime-lcm-canvas.git
%cd flowty-realtime-lcm-canvas
!pip install -r requirements.txt
!python ui.py --share

This noteworthy project is a community contribution from flowt.ai. If you find this project intriguing, explore more about flowt.ai and their array of compelling projects!