NVIDIA ACE: Bringing Digital Humans to Life with Generative AI
NVIDIA ACE is a powerful collection of technologies that empowers developers to create lifelike digital humans using generative AI. This suite includes ACE NIMs (NVIDIA Intelligent Microservices), which are microservices designed to function in cloud environments or on personal computers.
What Does ACE Offer?
NVIDIA provides various samples and reference applications through ACE NIMs, which are accessible via an evaluation license from NVIDIA AI Enterprise (NVAIE) through NGC (NVIDIA GPU Cloud).
To explore the capabilities of ACE, you can:
- Try NIM for creating digital humans.
- Get access to NVIDIA AI Enterprise.
- Download ACE Microservices for further exploration.
ACE Technologies
ACE comprises an array of advanced technologies that cater to different aspects of digital human creation:
- Riva Automatic Speech Recognition: Converts speech into text efficiently, suited for applications running on NVIDIA AI Enterprise.
- Riva Neural Machine Translation: Provides text translation capabilities.
- Riva Text-to-Speech: Transforms text into spoken words.
- Audio2Face: Converts audio into facial blendshapes for realistic lip-sync.
- AnimGraph: An animation controller for enhancing animation workflows.
- Omniverse RTX Rendering Microservice: A pixel streamer based on Omniverse for high-quality rendering.
- ACE Agent: A conversational controller supporting RAG workflows.
- Maxine Speech Live Portrait: Enables 2D lipsync and animation from images.
- Nemotron-3 4.5B SLM: Small language model for various linguistic tasks.
- Gaming Reference Workflow: Provides examples utilizing Audio2Face within the Unreal Engine.
- Customer Service Reference Workflow: Demonstrates customer service and kiosk solutions.
Key Benefits of ACE
-
State-of-the-Art Models and Microservices: Offers high-quality, real-time performance through NVIDIA's pre-trained models.
-
Safe and Consistent Results: AI models are trained on responsible and commercially safe data, ensuring accurate and appropriate outputs.
-
Flexible Deployment Options: Provides flexibility in deployment, whether it be on public or private clouds, Windows PCs, or a combination of both.
Digital Human Workflows
Developers can harness ACE's capabilities to build custom digital human solutions or opt to use NVIDIA's domain-specific AI workflows for applications like game characters, interactive digital assistants for customer service, and digital avatars for real-time communication.
Gaming Characters
The NVIDIA Kairos Sample is a project showing how to use Unreal Engine with the Audio2Face microservice to enhance game characters by integrating with Metahuman.
Customer Service
NVIDIA Tokkio is an AI-powered workflow that enables digital assistants to provide customer service in industries such as healthcare, finance, and retail by using cutting-edge language, speech, and animation generative AI models.
Resources and Tutorials
For those interested in diving deeper, NVIDIA provides comprehensive documentation and tutorials, covering components like Docker setup, Kubernetes installation, and various specific technologies such as NVIDIA Audio2Face, Riva ASR, and many more.
Additionally, example workflows demonstrate the practical application of ACE technologies through videos guiding on tasks such as Text-to-Gesture conversion, export of characters for specific platforms, and building stylized avatars.
NVIDIA ACE is licensed under the Apache 2 license and integrates with NGC microservices under the NVIDIA AI Product License, ensuring that users have the flexibility to work within their development environment while benefiting from industry-leading AI technologies.