At Computex 2024, attendees were treated to an engaging demonstration of NVIDIA’s latest innovation in digital human technology, showcased through the Covert Protocol tech and game demo. This collaboration with Inworld AI harnesses the power of NVIDIA ACE technologies to create lifelike in-game characters that possess unique personalities, knowledge, and relationships. Players embark on a detective-style adventure, utilizing text or voice commands to uncover specific information from these AI-generated characters.
Innovative Game Development
The demonstration provided a captivating glimpse into the future of game development, all powered by a high-performance GeForce RTX 40 Series GPU. My experience included extracting critical information from one of the characters by navigating through a series of puzzle-like steps, highlighting the potential of this technology in enhancing player engagement.
As the excitement continues, NVIDIA is set to unveil an in-game version of this technology at Gamescom 2024, featured in the upcoming multiplayer mech game, Mecha BREAK, developed by Amazing Seasun Games. This game is powered by NVIDIA’s groundbreaking Nemotron-4 4B NIM small language model, which promises to expedite the integration of Digital Humans into gaming experiences.
Mecha BREAK’s Digital Human character utilizes Audio2Face-3D NIM for dynamic facial animations, all processed on-device. Additionally, third-party models from Whisper facilitate speech recognition, while Elevenlabs provides realistic voice synthesis through cloud processing. This intricate blend of technologies allows players to engage with mechanic Martel, who offers insights on upcoming missions, mech load-outs, and customization advice.
Martel’s ability to respond to voice commands and suggest modifications enhances the gameplay experience, particularly for newcomers navigating the game’s extensive customization options. The Digital Human demo will be available for exploration on the Gamescom show floor, inviting players to interact with this innovative feature firsthand.
NVIDIA emphasizes the necessity of robust hardware for real-time performance, as the AI components demand significant processing power. However, the integration of cloud capabilities allows for efficient offloading of some processing tasks. Notably, the impact on frame rate when utilizing Digital Humans in Mecha BREAK is minimal—around 5%—comparable to adjusting texture quality from low to high. This impressive performance is achieved on a GeForce RTX 4090-powered rig, showcasing the high-end capabilities of this emerging technology in the realm of PC gaming.
For those eager to learn more about Mecha BREAK, additional details can be found on the game’s Steam page.