ebsynth_utility
Overview
Ebsynth_utility is a powerful extension designed to create videos using the combination of img2img and Ebsynth, without the need for Adobe After Effects (AE). It operates as an extension of the AUTOMATIC1111 User Interface (UI), providing users with the capability to output edited videos. For an optimized video editing experience, it is recommended to install Controlnet, which ensures all features of this extension function seamlessly. Controlnet supports video editing significantly, and pairing "canny" and "normal map" control nets is suggested for enhanced effectiveness.
Additionally, the extension includes a modified animatediff-cli tool for txt2video conversion, enabling users to specify prompts flexibly. Those interested can explore more about it via animatediff-cli-prompt-travel.
Example
Sample 1: Mask with clipseg
The demonstration showcases the following:
- The original video (first from left).
- A masked version excluding "finger" but including "cat" (second from left).
- A focused mask on the "cat head" (third from left).
- Color correction applied using color-matcher (right).
With this extension, multiple targets (e.g., cat, dog, boy, girl) can be specified and processed.
Sample 2: Blend Background
The tool allows blending videos with different themes:
- For the person: a blend of anime-style, high-quality screencaps.
- For the background: a cyberpunk factory/room ambience with anime style.
This blending technique can also be applied to any favorite videos providing a creative fusion of elements.
Sample 3: Auto Tagging
This feature automates tagging in all keyframes:
- Original video (left).
- Application of consistent prompts across all keyframes (center).
- Automatic tagging using deepdanbooru (right).
It enhances precise modifications such as facial expressions, tagging actions like "closed_eyes" and "hands_on_own_face" for detailed movements.
Sample 4: Auto Tagging with Dynamic Lora
Utilizing auto tagging and dynamically applying Lora, this feature adjusts prompts based on detected tags:
- Auto tagging (left).
- Auto tagging with "anyahehface" Lora added dynamically (right).
The extension adjusts settings based on tags, such as adding "smile" tags. A blacklist filters out incorrect tags.
Installation
Usage
- Access the [Ebsynth Utility] tab.
- Set up a project directory and fill in necessary fields like "Original Movie Path".
- Follow stages sequentially from 1 to 7, checking the console for progress completion.
Notes
Note 1: Editing Process
Example guidance is provided for editing a 1280x720, 30fps, 15-second video across stages, each detailing configurations and mode adjustments to ensure quality output.
Note 2: Multi-Controlnet Usage
Configuration instructions detail using multi-controlnet together, emphasizing settings in webui and controlnet options for img2img.
Note 3: Using Clipseg
Visual guidance is offered for incorporating clipseg within the project for enhanced masking tasks.
Overall, ebsynth_utility provides a robust framework for video editing enthusiasts looking to integrate img2img with a flexible and efficient UI without needing complex software like Adobe AE.