Microsoft has launched a limited preview of Copilot Vision, an innovative AI-powered feature that can understand and respond to questions about websites users are visiting using Microsoft Edge. This new tool is part of Copilot Labs, an opt-in program for experimental AI capabilities, and requires a subscription to Microsoft's Copilot Pro plan, which costs $20 per month.

Copilot Vision can analyze text and images on web pages to answer queries, summarize and translate text, and even handle tasks like highlighting discounted products in a store catalog. Additionally, it can serve as a game assistant, offering pointers during matches on Chess.com, for instance. According to Microsoft, when users enable Copilot Vision, it "sees the page you're on, it reads along with you, and you can talk through the problem you're facing together."

Notably, Microsoft is emphasizing user privacy with Copilot Vision. The company claims that the feature deletes data following every session, and processed audio, images, or text are not stored or used to train models. This cautious approach is likely a response to past AI privacy controversies and legal disputes with news outlets, such as The New York Times, which alleges that Microsoft allowed users to bypass its paywall using the Copilot chatbot on Bing.

Copilot Vision is currently limited in the types of websites it can view, with Microsoft blocking the feature from working on paywalled and "sensitive" content. The company has not defined what constitutes "sensitive" content, but it's likely to include pornographic or violent material. Furthermore, many major publishers have opted to block AI tools from trawling their websites, fearing data misuse and increased server costs. Microsoft claims that Copilot Vision will respect sites' "machine-readable controls on AI," but the company has not specified which controls it will adhere to.

Despite these limitations, Microsoft is committed to taking feedback from publishers to address their concerns. The company has collaborated with third-party publishers to understand how Vision can be used to help users better engage with and make decisions on their pages. This input will inform the design of Vision to ensure it interacts with web pages in a more helpful and respectful manner.

The future of Copilot Vision remains to be seen, but if the current trend holds, it may not work on some of the web's top news sites. Nevertheless, Microsoft's focus on user privacy and publisher concerns is a step in the right direction, and the company's willingness to take feedback and adapt its technology is a promising sign for the future of AI-powered web browsing assistants.