Google's DeepMind has published a comprehensive paper outlining its approach to ensuring the safety of Artificial General Intelligence (AGI), a type of AI that can perform any task a human can. The 145-page document, co-authored by DeepMind co-founder Shane Legg, predicts that AGI could arrive as early as 2030 and warns of potential "severe harm" and "existential risks" if proper safeguards are not implemented.

The paper's authors define AGI as a system that can match the capabilities of at least 99th percentile of skilled adults on a wide range of non-physical tasks, including metacognitive tasks like learning new skills. They anticipate the development of an "Exceptional AGI" before the end of the current decade, which could result in catastrophic consequences if not handled properly.

DeepMind's approach to AGI safety contrasts with that of other major AI labs, such as Anthropic and OpenAI. According to the paper, Anthropic places less emphasis on "robust training, monitoring, and security," while OpenAI is overly bullish on "automating" a form of AI safety research known as alignment research. DeepMind's paper also casts doubt on the viability of superintelligent AI, which can perform jobs better than any human, suggesting that significant architectural innovation would be required for such systems to emerge.

Instead, the authors propose that current paradigms will enable "recursive AI improvement," a positive feedback loop where AI conducts its own AI research to create more sophisticated AI systems. This, they argue, could be incredibly dangerous and highlights the need for techniques to block bad actors' access to hypothetical AGI, improve the understanding of AI systems' actions, and "harden" the environments in which AI can act.

While the paper acknowledges that many of the proposed techniques are nascent and have "open research problems," it emphasizes the importance of proactively planning to mitigate severe harms. "The transformative nature of AGI has the potential for both incredible benefits as well as severe harms," the authors write. "As a result, to build AGI responsibly, it is critical for frontier AI developers to proactively plan to mitigate severe harms."

Not everyone agrees with the paper's premises, however. Heidy Khlaaf, chief AI scientist at the nonprofit AI Now Institute, believes that the concept of AGI is too ill-defined to be "rigorously evaluated scientifically." Matthew Guzdial, an assistant professor at the University of Alberta, is skeptical about the possibility of recursive AI improvement, citing a lack of evidence for its feasibility.

Sandra Wachter, a researcher studying tech and regulation at Oxford, argues that a more pressing concern is AI reinforcing itself with "inaccurate outputs." She warns that the proliferation of generative AI outputs on the internet and the gradual replacement of authentic data could lead to models learning from their own outputs that are riddled with mistruths or hallucinations.

Despite the debates surrounding AGI, DeepMind's paper is a significant contribution to the ongoing discussion on AI safety. While it may not settle the debates over the feasibility of AGI, it highlights the need for responsible development and proactive planning to mitigate potential risks. As the AI field continues to evolve, it is crucial that developers, researchers, and policymakers work together to ensure that the benefits of AGI are realized while minimizing its potential harms.