Fay - Versatile Open-Source Digital Human Framework for Multiple Model Integration

Introducing the Fay Digital Human Framework

Overview

Fay is an innovative digital human framework designed to streamline the integration and deployment of digital human technologies in various applications. Fay aims to provide a robust interface that allows seamless interaction with digital human models and language models, while also ensuring easy accessibility for a range of devices—from microcontrollers to mobile applications and websites.

Key Features

Open Source and Free for Commercial Use: Fay is completely open-source, allowing businesses to use it commercially without any legal concerns.
Offline Accessibility: It supports fully offline operations, ensuring functionality without the need for constant internet connectivity.
Rapid Response: The system is designed to provide millisecond-level response times, making interactions quick and efficient.
Model Compatibility: Users can freely pair the framework with different digital human models, large language models, as well as ASR (Automatic Speech Recognition) and TTS (Text-to-Speech) systems.
Versatile Use Cases: Fay can automatically broadcast content such as virtual teacher sessions, virtual anchoring, or news reportage.
Cross-Platform Availability: It supports an array of platforms, including microcontrollers, apps, websites, large screens, and established systems.
Multi-User and Multi-Channel Concurrent Support: Fay is equipped to handle multiple users and channels simultaneously.
Comprehensive Interfaces: This includes text, voice, and digital human model interfaces, along with management control and automatic playback.
Flexible Command Interpretation: Users can configure and execute voice commands with ease.
Customizable Content Libraries: It allows the creation of personalized knowledge bases, Q&A pairs, and character information.
Conversation Management: Fay supports dialogue awakening and interruption functions.
Operational Modes: It can run in both server and standalone modes, offering additional versatility.
Expressive Output: The framework supports robotic expressions for a more interactive experience.

Getting Started

Environment

Fay works on Python versions 3.9 to 3.12 and is compatible with Windows, macOS, and Linux operating systems.

Installation

To get started, you need to install the necessary dependencies:

pip install -r requirements.txt

Configuration

Modify the ./system.conf file according to your setup needs.

Running Fay

To start the Fay controller:

python main.py

Docker Option

If preferred, Fay can be launched using Docker:

Download the Assistant Edition from the repository.
Modify the ./system.conf file.
Remove pyqt5~=5.15.6 from requirements.txt.

Then build and run the Docker container:

docker build -t fay ./fay-assistant-edition

docker run -it --rm -p 5000:5000 -p 10001:10001 -p 10002:10002 -p 10003:10003 fay

Advanced Usage

Using Digital Humans (Optional)

Explore integration with various platforms such as UE, Unity, 2D Metahuman Stream, Duix for Android, and Aibote for Windows-based clones using the respective resources provided.

Product Integration (Optional)

For API access and further integration guidelines, refer to the specified documentation.

Contact and Community

For business inquiries, contact: 467665317 (QQ)

Join the community for more tutorials and support: Follow the WeChat public account fay数字人 (ensure to star the repository first).

WeChat Public Account