Researchers from TikTok owner ByteDance have unveiled a groundbreaking AI system, OmniHuman-1, capable of generating deepfake videos that are arguably the most realistic to date. The system's demos have left experts and observers alike in awe, but also raised significant concerns about the potential misuse of such technology.

Deepfaking AI is no longer a novelty, with numerous apps and tools available that can insert someone into a photo or make a person appear to say something they didn't actually say. However, most deepfakes, particularly video deepfakes, often fail to clear the uncanny valley, exhibiting telltale signs of AI involvement. OmniHuman-1, on the other hand, has demonstrated an unprecedented level of realism, making it increasingly difficult to distinguish between real and fake videos.

The ByteDance team has showcased several impressive examples of OmniHuman-1's capabilities, including a fictional Taylor Swift performance, a TED Talk that never took place, and a deepfaked Einstein lecture. According to the researchers, the system only requires a single reference image and audio, such as speech or vocals, to generate a video. The output video's aspect ratio and subject's "body proportion" can also be adjusted, allowing for a high degree of customization.

OmniHuman-1 can even edit existing videos, modifying the movements of a person's limbs to create a seamless fake clip. While the system is not perfect, with limitations including the quality of reference images and certain poses, its capabilities are undeniably impressive and have significant implications for the future of deepfake technology.

The rise of deepfakes has already led to numerous instances of misinformation and fraud. Last year, political deepfakes spread rapidly around the globe, with examples including a Chinese Communist Party-affiliated group posting AI-generated audio of a politician in Taiwan and deepfake videos depicting the president of Moldova resigning. In South Africa, a deepfake of rapper Eminem supporting a local opposition party circulated ahead of the country's election.

Deepfakes are also being increasingly used to carry out financial crimes, with consumers being duped by fake celebrities offering fraudulent investment opportunities and corporations being swindled out of millions by deepfake impersonators. According to Deloitte, AI-generated content contributed to more than $12 billion in fraud losses in 2023, with projected losses reaching $40 billion in the U.S. by 2027.

In response to the growing concerns surrounding deepfakes, hundreds of AI experts signed an open letter last February calling for strict regulation. While there is currently no federal law criminalizing deepfakes in the U.S., over 10 states have enacted statutes against AI-aided impersonation. California's law, which is currently stalled, would empower judges to order the removal of deepfakes or impose monetary penalties on those responsible.

Despite efforts to limit the spread of deepfakes, the volume of such content online continues to grow at an alarming rate. A May 2024 survey by ID verification firm Jumio found that 60% of people encountered a deepfake in the past year, with 72% of respondents expressing concern about being fooled by deepfakes on a daily basis. A majority of those surveyed also supported legislation to address the proliferation of AI-generated fakes.

As the capabilities of deepfake technology continue to advance, it is essential that policymakers, tech companies, and individuals take proactive steps to mitigate the risks associated with this technology. The development of OmniHuman-1 serves as a stark reminder of the need for vigilance and cooperation in the face of rapidly evolving AI capabilities.