At Computex 2023 in Taipei, Nvidia CEO Jensen Huang unveiled ACE for Games, a groundbreaking development in the world of gaming and artificial intelligence. During the keynote, Huang introduced NVIDIA Avatar Cloud Engine (ACE) for Games, a custom AI model foundry service designed to bring intelligent non-playable characters (NPCs) to life through natural language interactions. This innovative technology enables game characters to engage in lifelike conversations with players, revolutionizing interactivity and immersion in gaming.
By leveraging generative AI and advanced technologies such as language models, speech recognition, text-to-speech, and facial animation, game developers can create realistic and dynamic characters that respond to players’ queries and actions in a manner consistent with their narrative backstories.
Table of Contents
Power of NVIDIA NeMo
One of the key components of ACE for Games is NVIDIA NeMo, a platform that enables developers to build, customize, and deploy language models specific to their games. These large language models can be tailored with character lore and backstories, ensuring coherent and engaging interactions. Furthermore, NeMo Guardrails protect against unproductive or unsafe conversations, enhancing the player’s experience and maintaining a safe gaming environment.
Realistic Conversations with AI NPCs:
By integrating NVIDIA Riva automatic speech recognition and text-to-speech capabilities, players can engage in live speech conversations with game characters. This advancement enables natural language queries to be processed in real-time, with NPCs delivering real-sounding responses that match their unique personalities and narratives.
Expressive Facial Animation with Omniverse Audio2Face
NVIDIA Omniverse Audio2Face, another component of ACE for Games, allows developers to create expressive facial animations for game characters. By utilizing the power of AI and the Omniverse connectors for Unreal Engine 5, facial animations can be dynamically generated to synchronize with the speech track, further enhancing the realism and immersion of the gaming experience.
Collaboration with Convai
NVIDIA collaborated with Convai, an NVIDIA Inception startup, to showcase the capabilities of ACE for Games. During the game demo showcased by Huang, player had the opportunity to interact with an NPC named Jin, who serves as the owner of a ramen noodle shop. Using voice commands, the human player asked questions, and Jin provided responses that were remarkably authentic and aligned with the NPC’s intricate backstory.
Player: Hey Jin, how are you?
Jin: Unfortunately not so good.
Player: How come?
Jin: I am worried about the crime around here. It’s gotten bad lately. My ramen shop got caught in the crossfire.
Player: Can I help?
Jin: If you want to do something about this, I have heard rumors that the powerful crime lord Kumon Aoki is causing all kinds of chaos in the city. He may be the root of this violence.
Player: I’ll talk to him, where can I find him?
Jin: I have heard he hangs out in the underground fight clubs on the city’s east side. Try there.
Player: OK, I’ll go.
Jin: Be careful, Kai.
Deploy NVIDIA ACE for Gaming Models Locally or in the Cloud
The neural networks enabling NVIDIA ACE for Games are optimized for different capabilities, with various size, performance, and quality trade-offs. The ACE for Games foundry service will help developers fine-tune models for their games, then deploy via NVIDIA DGX Cloud, GeForce RTX PCs, or on-premises for real-time inferencing. The models are optimized for low latency, ensuring immersive and responsive interactions in games.
NVIDIA’s ACE for Games showcases their dedication to pushing the boundaries of AI in gaming and paves the way for a future where lifelike conversations with game characters are the norm.
Source: NVIDIA Newsroom
