Nvidia has announced a breakthrough in AI audio generation with its new tool, Fugatto, which can create "sounds never heard before" based on text prompts. This innovative technology has the capability to generate and edit music, speech, or sounds using inputs it has never been trained on, making it a game-changer in the field of audio production.

Fugatto's capabilities are demonstrated in a video showcasing its ability to create songs based on wild prompts, such as "Create a saxophone howling, barking, then electronic music with dogs barking." The tool can also produce unique sound effects based on descriptive text, like "Deep, rumbling bass pulses paired with intermittent, high-pitched digital chirps, like the sound of a massive sentient machine waking up."

One of the most impressive features of Fugatto is its ability to transform the sound of someone's voice, changing their accent or tone to convey emotions like anger or calmness. Additionally, the tool can edit music by isolating vocals, adding instruments, and even changing melodies by swapping out instruments, such as replacing a piano with an opera singer.

A paper released alongside the announcement reveals the extensive list of datasets used to train Fugatto, including a library of sound effects from the BBC. This training enables the tool to perform a wide range of tasks with high accuracy, without requiring additional data.

Fugatto stands out from other AI audio tools available from companies like Stability AI, OpenAI, Google DeepMind, ElevenLabs, and Adobe, which do not claim to create completely new and unheard-of sounds. However, some AI startups are facing copyright lawsuits over their music creation tools, and a recent report found that Nvidia and other companies trained AI models on subtitles from thousands of YouTube videos.

The development of Fugatto required the creation of a massive dataset with millions of audio samples, followed by the development of instructions that expanded the model's capabilities while achieving more accurate performance. Although Nvidia has not announced a release date for Fugatto, its potential impact on the music and audio production industries is significant.

The implications of Fugatto's capabilities are far-reaching, with potential applications in music composition, sound design, and even voice acting. As AI audio generation technology continues to evolve, it will be interesting to see how it is adopted and utilized by creatives and industries alike.

In conclusion, Nvidia's Fugatto represents a significant breakthrough in AI audio generation, offering unparalleled capabilities and possibilities for creatives and industries. As the technology continues to advance, it will be exciting to see the innovative applications that emerge from this groundbreaking tool.