NVIDIA Avatar Cloud Engine

NVIDIA Avatar Cloud Engine (ACE) is a comprehensive suite of technologies designed to bring digital humans, AI non-player characters (NPCs), and interactive avatars to life using generative AI. This software enables developers and enterprises to create realistic and interactive digital characters for applications in gaming, customer service, and video conferencing.

Key features:

- State-of-the-Art Models: Incorporates NVIDIA Riva for speech recognition, text-to-speech, and translation; NVIDIA NeMo for customizable generative AI models; NVIDIA Audio2Face for lifelike facial animation; and NVIDIA RTX for advanced rendering.
- Flexible Deployment: Supports both cloud and local deployment options, including optimized containers for various environments.
- Safety and Consistency: Utilizes commercially safe data and offers fine-tuning options to maintain appropriate responses.
- Real-time Performance: Delivers high-quality, real-time performance for interactive applications.

How it works:

1. Developers integrate ACE technologies into their applications.
2. The system processes user input through speech recognition or text.
3. Generative AI models produce appropriate responses.
4. Audio2Face synchronizes facial animations with generated speech.
5. RTX technologies render the digital character in real-time.

Integrations:

Unreal Engine 5, Inworld AI, UneeQ, Studio Wildcard, Hippocratic AI, NVIDIA Maxine

Use of AI:

ACE utilizes generative AI models for natural language processing, speech synthesis, and facial animation. These models enable the creation of dynamic, context-aware digital characters that can engage in natural conversations and respond to user inputs in real-time.

AI foundation model:

The system is built on NVIDIA's advanced AI models, including NVIDIA Riva for speech and language processing, NVIDIA NeMo for language generation and customization, and NVIDIA Audio2Face for facial animation. These models are trained on NVIDIA supercomputers using responsibly licensed data.

Target users:

- Game developers
- Customer service providers
- Healthcare professionals
- Enterprises seeking to enhance digital human interactions

How to access:

NVIDIA ACE is available as a web application, API, and SDK. Developers can apply for early access to new features through NVIDIA's Early Access Program.