Ilya Sutskever, co-founder and former chief scientist of OpenAI, made a rare public appearance at the Conference on Neural Information Processing Systems (NeurIPS) in Vancouver, where he dropped a bombshell: the way AI is built is about to change dramatically. According to Sutskever, the industry has reached "peak data," meaning that there will be no more new data to train on, and this will force a shift away from the current model development approach.

Sutskever's statement is significant, given his influence in the AI community and his role in shaping the direction of AI research. He believes that existing data can still take AI development farther, but the industry is tapping out on new data to train on. This dynamic will eventually force a shift away from the way models are trained today. Sutskever compared the situation to fossil fuels, stating that just as oil is a finite resource, the internet contains a finite amount of human-generated content.

The implications of this shift are far-reaching. Sutskever predicts that next-generation models will be "agentic in a real way," meaning they will be autonomous AI systems that perform tasks, make decisions, and interact with software on their own. These systems will also be able to reason, unlike today's AI, which mostly pattern-matches based on what a model has seen before. According to Sutskever, future AI systems will be able to work things out step-by-step in a way that is more comparable to human thinking.

Sutskever's vision for the future of AI is both exciting and unsettling. He notes that the more a system reasons, the more unpredictable it becomes. He compared the unpredictability of "truly reasoning systems" to how advanced AIs that play chess are unpredictable to the best human chess players. However, he also believes that these systems will "understand things from limited data" and "will not get confused."

Sutskever's talk also touched on the scaling of AI systems, drawing a comparison with evolutionary biology. He cited research that shows the relationship between brain and body mass across species, noting that hominids (human ancestors) show a distinctly different slope in their brain-to-body mass ratio on logarithmic scales. He suggested that, just as evolution found a new scaling pattern for hominid brains, AI might similarly discover new approaches to scaling beyond how pre-training works today.

When asked how researchers can create the right incentive mechanisms for humanity to create AI in a way that gives it "the freedoms that we have as homosapiens," Sutskever responded cautiously, saying that he doesn't "feel confident answering questions like this" because it would require a "top-down government structure." However, he did encourage speculation on the topic, suggesting that maybe having AIs with rights wouldn't be a bad end result.

Sutskever's predictions and insights have significant implications for the AI industry and beyond. As the industry continues to grapple with the challenges of developing more advanced AI systems, his words serve as a reminder that the current approach may not be sustainable in the long run. The question now is: what's next for AI development, and how will the industry adapt to this new reality?