Sony is venturing into uncharted territory with its latest experiment: creating AI-powered characters for its PlayStation games. An internal video leaked to The Verge showcases an AI-powered version of Aloy, the protagonist from the popular Horizon series, capable of holding conversations with players through voice prompts during gameplay.

The demo, narrated by Sharwin Raghoebardajal, director of software engineering at Sony Interactive Entertainment, demonstrates Aloy's ability to respond to queries with an AI-powered synthesized voice and facial movements. This technology is powered by OpenAI's Whisper for speech-to-text, GPT-4 and Llama 3 for conversations and decision-making, and Sony's internal Emotional Voice Synthesis (EVS) system for speech generation. Additionally, audio-to-face animation is enabled by Sony's Mockingbird technology.

While the demo runs on PC, Raghoebardajal revealed that Sony has also experimented with running parts of this technology directly on PS5 consoles with "little overhead." This technology was first demonstrated internally a year ago and was showcased behind closed doors at Sony's Technology Exchange Fair (STEF) in Tokyo last November.

The implications of this technology are far-reaching, with potential consequences for voice actors and game developers. As Raghoebardajal notes, "This is just a glimpse of what is possible," leaving many questions unanswered, such as the feasibility of conversing with Aloy while playing as her in Horizon Forbidden West, and the potential impact on the creative process of making video games.

This development comes amidst a growing trend in the gaming industry, with companies like Nvidia and Microsoft exploring similar AI-powered technologies. Nvidia's Ace technology, demonstrated throughout 2024, allows players to speak freely to video game characters, while Microsoft is partnering with Inworld AI to bring AI characters to Xbox, enabling game developers to use generative AI characters, storylines, and more.

The gaming industry is already grappling with the impact of AI on game development, with nearly half (49%) of 3,000 respondents to a Game Developers Conference (GDC) 2024 survey reporting the use of generative AI tools in their workplace. This news is likely to spark further debate on the role of AI in video game creation, particularly ahead of the GDC conference in San Francisco next week.

Sony did not respond to requests for comment on the leaked internal video. As the gaming industry continues to evolve, it remains to be seen how AI-powered characters will shape the future of video game development and the creative process.