x-stable-diffusion

Introduction to x-stable-diffusion

Overview

The x-stable-diffusion project, developed by Stochastic, is designed to accelerate image generation using the Stable Diffusion model. This project is a bundle of various techniques that not only enhance the speed of image generation but also optimize for cost-efficiency. With the availability of example images and comprehensive benchmarks, users can easily select the most suitable method for their specific requirements.

The project is supported by a command-line interface (CLI) called stochasticx, which simplifies the deployment process on local machines. The x-stable-diffusion project demonstrates the potential improvements in performance and cost-effectiveness for image generation tasks.

Installation

Quickstart

To get started with x-stable-diffusion, ensure that Python and Docker are installed on your system. Follow these steps:

1. Install the stochasticx library:

   pip install stochasticx
   

2. Deploy the Stable Diffusion model:

   stochasticx stable-diffusion deploy --type aitemplate
   

3. Perform inference with the deployed model:

   stochasticx stable-diffusion inference --prompt "Riding a horse"
   

   You can explore other options for the inference command by using:

   stochasticx stable-diffusion inference --help
   

4. Monitor deployment logs:

   stochasticx stable-diffusion logs
   

5. Stop and remove the deployment:

   stochasticx stable-diffusion stop
   

Achieving Less Than 1s Latency

For faster performance, adjust the model’s settings:

• Change num_inference_steps to 30 for images generated in under one second.

{
  'max_seq_length': 64,
  'num_inference_steps': 30, 
  'image_size': (512, 512) 
}

Running on Google Colab

Stochastic offers Google Colab notebooks for testing x-stable-diffusion, allowing users to run the entire workflow using a T4 GPU. Users can try different settings such as PyTorch with FP16 precision or TensorRT.

Manual Deployment

If you prefer not using the CLI, there are manual methods available for deployment in directories such as:

• AITemplate
• FlashAttention
• nvFuser
• PyTorch
• TensorRT

Optimizations

The project features several optimization technologies to enhance performance:

• AITemplate: A framework by Meta that optimizes image generation.
• TensorRT: NVIDIA's solution for high-performance deep learning inference.
• nvFuser: Integrates with PyTorch for improved computation efficiency.
• FlashAttention: Used within the Xformers library to boost attention mechanisms in neural networks.

Benchmarks

Benchmarks were conducted using an A100 GPU with CUDA 11.6. The trials averaged over 50 iterations using specific parameters, such as num_inference_steps set to 50.

Performance on Different GPUs

1. A100 GPU:

   The latency and GPU VRAM usage results showed that the AITemplate framework delivers the best balance of speed (1.38s) and memory usage (4.83GB).

2. T4 GPU:

   The TensorRT framework had the lowest latency among supported techniques, making it a favorable choice for users leveraging a T4 GPU.

3. Batched Results:

   Tests with batch sizes from one to twenty-four were performed, showcasing varying latency and VRAM requirements across different frameworks.

Sample Generated Images

The project has generated a diverse set of images, each created using different optimization technologies, enhancing creativity and application flexibility for various prompts.

References and Community

The project draws from renowned libraries and frameworks, such as HuggingFace Diffusers and AITemplate. A vibrant community supports the project on Discord, and the development team encourages contributions to enhance features and documentation.

For those interested in managed hosting options on cloud platforms, Stochastic provides dedicated solutions that integrate seamlessly with their offerings.

By leveraging x-stable-diffusion, users can experience significant improvements in their image generation tasks in terms of both time efficiency and cost savings.