High Resolution Depth Maps for Stable Diffusion WebUI
The stable-diffusion-webui-depthmap-script is an innovative add-on crafted for AUTOMATIC1111's Stable Diffusion WebUI. This unique script introduces an exciting feature: the ability to create high-resolution depth maps from images. Depth maps are crucial in visual computing as they represent the distance between objects in a scene and the camera capturing them, enhancing the image with insightful depth information.
Key Features
-
Depth Map Creation: Users can generate depth maps from both generated and custom images with this software. Once created, these maps can be used to produce 3D stereo image pairs, including side-by-side and anaglyph images, which showcase visually compelling 3D effects.
-
3D Outputs: Not limited to depth visualization, the script can produce additional outputs like normal maps and three-dimensional meshes, which are vital for advanced rendering and 3D modeling tasks. These outputs can be directly viewed or integrated as assets in 3D engines.
-
Compatibility and Versatility: The program integrates seamlessly with Rembg, a background removal tool, and supports batch processing and video management. Furthermore, it is adaptable enough to run in standalone mode, independent of the Stable Diffusion WebUI.
Technical Infrastructure
The creation of realistic depth maps is grounded in models sourced from several esteemed research repositories such as Marigold, MiDaS, and ZoeDepth by Intel ISL, and LeReS from AdelaiDepth. These models are the backbone for generating robust depth information, and high-resolution depth maps are achieved using the BoostingMonocularDepth algorithm.
Additionally, the script includes stereoscopic image generation through a custom algorithm and employs a technique called 3D Photography using Context-aware Layered Depth Inpainting for creating 3D inpainted meshes and video renders.
Installation and Updates
-
As an Extension: The depth map script can be easily installed as an extension from the Stable Diffusion WebUI. Users can find it under the Extensions tab where they can check for updates, apply changes, and restart the interface when needed.
-
Standalone Use: Users looking to operate the script independently can clone the repository, install the necessary requirements, and run it using
main.py.
Usage Guide
Once installed, users can select the "DepthMap" script from a selection box in either txt2img or img2img modes. Multiple models are available for computation on GPUs or CPUs, allowing flexibility based on available resources. Users can control the depthmap's attributes, such as resolution, clipping, inversion, and saving preferences. The script also supports stereo image generation and video creation from 3D inpainted meshes, offering various settings for customizing the 3D effect, balance, and gap fill techniques.
Contributions and Acknowledgements
The project invites developers to contribute by fixing bugs and introducing new features. It recognizes the advancements from numerous academic works and software, ensuring a robust backbone for its reliable performance.
In summary, this script is a powerful tool for users seeking to enhance their visual outputs with depth information and 3D capabilities, supported by a comprehensive infrastructure and easy usability.