In a groundbreaking achievement, researchers from Stanford and the University of Washington have managed to create a low-cost AI reasoning model that rivals OpenAI's, in a remarkably short time of just 26 minutes, and at a cost of under $50. This feat, outlined in a paper published last week, has significant implications for the AI industry, which has long been dominated by expensive and time-consuming model development.

The researchers employed a technique called distillation, which allows smaller models to draw from the answers produced by larger ones. In this case, they used Google's Gemini 2.0 Flash Thinking Experimental AI reasoning model as the basis for their own model, called s1. Notably, Google's terms of service prohibit using Gemini's API to develop competing AI models, although the researchers did not directly access the API.

The s1 model was based on Qwen2.5, an open-source model from Alibaba Cloud. Initially, the researchers trained the model on a pool of 59,000 questions, but found that a smaller dataset of just 1,000 questions yielded similar results, with no substantial gains from the larger dataset. The training process utilized 16 Nvidia H100 GPUs, highlighting the efficiency of the approach.

The s1 model also incorporates a technique called test-time scaling, which enables the model to "think" for a longer period before producing an answer. By adding "Wait" to the model's response, the researchers forced the model to double-check its answer, often correcting incorrect reasoning steps. This approach is similar to that used by OpenAI's o1 reasoning model.

The rise of smaller and cheaper AI models, like s1, threatens to disrupt the AI industry's status quo. If proven scalable, these models could demonstrate that major companies like OpenAI, Microsoft, Meta, and Google do not need to invest billions of dollars in training AI models, nor maintain massive data centers filled with thousands of Nvidia GPUs. This could lead to a significant shift in the industry's development paradigm.

The achievement is also notable in the context of the ongoing controversy surrounding DeepSeek, a startup accused by OpenAI of distilling information from its models to build a competitor, violating its terms of service. The researchers behind s1 claim that their model exceeds OpenAI's o1-preview on competition math questions by up to 27%.

As the AI landscape continues to evolve, the emergence of smaller, more affordable models like s1 is likely to have far-reaching implications for the industry. With the potential to democratize access to AI development, these models could unlock new opportunities for innovation and entrepreneurship.