clipsai

Introducing ClipsAI

ClipsAI is an innovative open-source Python library designed to simplify the process of converting lengthy videos into shorter, more manageable clips. With just a few lines of code, ClipsAI enables users to segment videos and change their aspect ratio from 16:9 to a more social media-friendly format of 9:16.

Purpose and Target Use

ClipsAI is particularly effective for audio-centric, narrative-driven content such as podcasts, interviews, speeches, and sermons. It utilizes video transcripts to identify and generate clips intelligently. The library's resizing algorithm adjusts the frame to focus dynamically on the current speaker, thus maintaining visual engagement while altering aspect ratios.

For those interested in exploring further, comprehensive documentation is available on the ClipsAI Documentation website, and demos showcasing clips generated using this library can be viewed on the UI demo page.

Installation Guidelines

To get started with ClipsAI, follow these basic installation steps:

1. Install Python Dependencies: It is recommended to work within a virtual environment to prevent dependency conflicts. This can be done using environments like venv.

   Execute the following commands:

   pip install clipsai
   

   pip install whisperx@git+https://github.com/m-bain/whisperx.git
   

2. Install libmagic: Follow the instructions provided in the libmagic repository.

3. Install ffmpeg: Guidelines are available here.

Creating Clips

To create clips using ClipsAI, the video transcription is essential. The process utilizes WhisperX, which is an enhanced version of the Whisper transcription tool. WhisperX specifically tracks the start and stop times for each word spoken in the video.

Here's a simple guide on how to create clips:

from clipsai import ClipFinder, Transcriber

transcriber = Transcriber()
transcription = transcriber.transcribe(audio_file_path="/abs/path/to/video.mp4")

clipfinder = ClipFinder()
clips = clipfinder.find_clips(transcription=transcription)

print("StartTime: ", clips[0].start_time)
print("EndTime: ", clips[0].end_time)

Resizing Videos

Resizing videos in ClipsAI involves using a Hugging Face access token for speaker diarization, utilizing the Pyannote software. While this may sound complex, rest assured that the service is free and easy to access via the Pyannote HuggingFace page.

Below is a simple script to resize videos:

from clipsai import resize

crops = resize(
    video_file_path="/abs/path/to/video.mp4",
    pyannote_auth_token="pyannote_token",
    aspect_ratio=(9, 16)
)

print("Crops: ", crops.segments)

ClipsAI is a powerful tool for anyone working with narrative video content, enabling efficient creation and customization of video clips suitable for various platforms and purposes. With ClipsAI, the complexity of video editing is significantly reduced, allowing more focus on creating engaging content.