opendream

Opendream: A Web UI for Creativity

Opendream is designed to enhance and simplify the creative process in Stable Diffusion workflows by offering features like layering, non-destructive editing, and easy extension integration. For a quick overview, check out the demo video.

Getting Started

Before diving into Opendream, ensure Node is installed on your machine, which you can download here. To begin using Opendream:

1. Clone the Opendream repository to your computer.
2. Open your terminal, navigate to the Opendream project directory, and execute sh ./run_opendream.sh. This command will start both the frontend and backend in about 30 seconds.

Features of Opendream

Diffusion models are powerful tools for generating and manipulating images, yet they often seem complex and challenging to control. Typically, interfaces for these models handle image tasks like text-to-image (text2img) generation or image editing in a destructive way, where new edits overwrite previous ones. This limits the freedom to experiment and build upon existing images.

Layering and Non-destructive Editing

Opendream addresses this by allowing non-destructive editing, which means users can edit images without losing the original content. This feature uses layers and masks, similar to those in Photoshop. You can add, modify, or remove layers without affecting the underlying image data. This capability ensures creativity can flourish without limits, as you can revisit earlier stages of your work at any time.

Save and Share Workflows

With Opendream, workflows—the steps and processes within an image project—can be saved in a file and reopened later or shared with collaborators. This file contains all information about the layers and the actions taken, ensuring transparency and reproducibility in creative processes.

Extensions: Simple and Flexible

As the open-source community continues to grow, extending capabilities becomes crucial. Opendream offers a simple method for writing and installing extensions. Unlike some other applications where extensions can be complicated to use, Opendream simplifies this process.

Extensions can be as straightforward as writing a Python function, enabling users to tailor the tool to their needs. It even allows for integration with external APIs, like OpenAI's DALL-E, making it adaptable for various artistic workflows.

Available Extensions

Opendream supports several key extensions, including:

• Dream and mask_and_inpaint: Core operations for image manipulation.
• Instruct Pix2Pix: For instructional image edits.
• ControlNet Canny and ControlNet Openpose: Advanced editing features.
• Segment Anything (Sam): For precise image segmentation.

These extensions can be installed directly through the Web UI or manually by adding them to the opendream/extensions folder. Each extension may have specific requirements, so check the requirements.txt file for additional dependencies.

Creating Your Own Extension

To create a custom extension:

1. Add a new Python file to the opendream/extensions directory.
2. Define a function with the @opendream.define_op decorator. This will register the function with the Opendream backend and allow it to be used as an extension.

Ensure your function parameters are type-hinted, as this enables seamless integration into the system, converting inputs into forms on the frontend.

Contributions and Licensing

Opendream is a collaborative project developed by Varun Shenoy, Eric Lou, Shashank Rammoorthy, and Rahul Shiv under Stanford's CS 348K course. The project welcomes contributions from the community and is licensed under the MIT License, making it accessible and open for modification and distribution.