The rapid progress of large language models (LLMs) has been nothing short of remarkable, with recent breakthroughs like DeepSeek promising unprecedented speed and affordability. However, a critical flaw persists: these models' inability to provide consistently accurate answers. This limitation poses a significant barrier to their adoption in areas where right-or-wrong answers are essential, such as fact-checking, decision-making, and automation of critical tasks.

As analyst Benedict Evans astutely observes, "There is also a broad class of task that we would like to be able to automate, that's boring and time-consuming and can't be done by traditional software, where the quality of the result is not a percentage, but a binary." In other words, for certain tasks, the answer is not better or worse, but simply right or not right. Until generative AI can deliver facts rather than probabilities, its potential will remain unrealized in many domains.

The recent excitement surrounding DeepSeek, which sent "shock waves" through the AI community, may be short-lived. While the model's speed and cost improvements are undeniable, the industry's focus on performance and affordability may be misplaced. As Evans notes, "Every week there's a better AI model that gives better answers... But a lot of questions don't have better answers, only right answers, and these models can't do that."

The application layer, rather than infrastructure, is where the most interesting and important developments in generative AI are happening. However, many applications rely on right-or-wrong answers, not probabilistic outputs based on patterns observed in training data. This fundamental limitation makes these models potentially worse than useless for certain tasks, as they can be exceptionally confident and wrong simultaneously.

Evans' example of trying to find the number of elevator operators in the United States in 1980, using a generative AI model, illustrates this point. Despite providing the model with the correct primary source, he received a range of incorrect answers, demonstrating the model's inability to deliver a simple, verifiable fact. This lack of accuracy is a major concern, as it undermines trust in these systems and limits their potential in critical applications.

However, there may be cases where the error rate of generative AI models becomes a feature rather than a bug. As Evans poses, "Are there places where [generative AI's] error rate is a feature, not a bug?" This question encourages us to rethink our approach to programming and accept probability rather than certainty as a desirable outcome. By doing so, we may uncover new ways to harness the potential of generative AI, even if it means redefining what we consider "accurate" or "correct."

In conclusion, while generative AI has made tremendous strides in recent years, its accuracy limitations pose a significant hurdle to widespread adoption. To unlock the true potential of these models, the industry must focus on developing systems that can provide reliable, verifiable answers, rather than simply relying on speed and affordability. Only then can we begin to realize the transformative power of generative AI in a wide range of applications.