ComfyUI Inpaint Nodes
The ComfyUI Inpaint Nodes project enhances the inpainting capabilities of ComfyUI by integrating various models and tools such as the Fooocus inpaint model for SDXL, LaMa, MAT, and pre-filling techniques for inpaint and outpaint areas. These resources offer a streamlined approach to fill and expand incomplete areas within an image.
Fooocus Inpaint
Fooocus inpaint is a specialized model that can be applied to SDXL checkpoints to create an inpainting model. This model enables seamless filling and expansion areas in an image, similar to other inpaint models. To use it, one must download the models from lllyasviel/fooocus_inpaint and save them in the ComfyUI/models/inpaint directory. It's crucial to use regular checkpoints rather than distilled versions like Turbo or Lightning.
Inpaint Conditioning
ComfyUI supports using the Fooocus inpaint model directly with its VAE Encode (for Inpainting). However, this usage does not allow combining masked areas with existing content if the denoise strength is less than 1.0. The addition of the InpaintModelConditioning node allows for combining models with existing content, although the latent result cannot be directly used with the Apply Fooocus Inpaint. Instead, a new node, VAE Encode & Inpaint Conditioning, helps to achieve this by providing outputs for further processing.
Inpaint Pre-processing
Pre-processing nodes are available for enhancing masked areas before inpainting. They help avoid seams as long as the mask is adequately large:
- Expand Mask: Modifies the mask area by a set pixel amount and applies a smoothening blur at the edges.
- Fill Masked: Offers three fill modes—neutral, telea, and navier-stokes—for filling the masked area with varied smooth transitions.
- Blur Masked: Blurs images into masked areas, keeping color consistency throughout.
Additional models like LaMa and MAT can be used for inpainting. These models enable efficient outpainting or object removal by loading with Load Inpaint Model and applying with Inpaint (using Model) nodes.
Inpaint Post-processing
The Denoise to Compositing Mask node is beneficial for using masks with variable denoise strengths in ComfyUI's "Differential Diffusion" node. It adjusts mask values to create a compositing mask suitable for post-processing blending.
Example Workflows
The project offers several example workflows to demonstrate its capabilities:
- Simple: A basic workflow for complete content replacement.
- Refine: Advanced usage for refining existing content with variable denoise strengths.
- Outpaint: Dedicated to outpainting using pre-processing techniques.
- Pre-process: Complex builds for experimenting with various pre-processing methods.
- Promptless: A workflow akin to above but omitting text prompts, relying on the IP-Adapter.
Installation
To install ComfyUI Inpaint Nodes, users can utilize the ComfyUI Manager, clone the repository into ComfyUI/custom_nodes, or directly download it. Ensuring OpenCV's installation is necessary for telea and navier-stokes fill modes.
pip install opencv-python
Acknowledgements
The project is built upon the work of several contributors:
- Fooocus Inpaint: lllyasviel/Fooocus
- LaMa/Advimman: lama
- MAT/Fenglinglwb: MAT
- LaMa/MAT Implementation: chaiNNer-org/spandrel
This suite of nodes significantly enhances the inpainting capabilities within ComfyUI, providing users with a robust set of tools for image manipulation tasks.