Microsoft’s AI can now read your screen — or rather, the websites you’re browsing.
On Thursday, the company began rolling out a limited, U.S.-only preview of Copilot Vision, a tool that can understand and respond to questions about sites you’re visiting using Microsoft Edge. Gated behind Copilot Labs, an opt-in program for experimental AI capabilities, Copilot Vision can analyze text and images on web pages to answer queries like “What’s the recipe for this lasagna?”
Note that Copilot Labs requires a subscription to Microsoft’s Copilot Pro plan, which costs $20 per month.
Beyond answering questions, Copilot Vision can summarize and translate text, and handle tasks like spotlighting discounted products in a store catalog. It can also serve as a game assistant, for example offering pointers during matches on Chess.com.
“When you choose to enable Copilot Vision, it sees the page you’re on, it reads along with you, and you can talk through the problem you’re facing together,” Microsoft wrote in the blog post shared with TechCrunch. “It’s a new way to invite AI along with you as you navigate the web, tucked neatly into the bottom of your Edge browser whenever you want to ask for help.”
No doubt eager to avoid more bad press from AI privacy fumbles, Microsoft is stressing that Copilot Vision deletes data following every session. Processed audio, images or text aren’t stored or used to train models, the company says — at least not in this preview release.
Copilot Vision is also limited in the types of websites it can view. For the time being, Microsoft’s blocking the feature from working on paywalled and “sensitive” content, limiting Vision to a pre-approved list of “popular” sites.
Leave a Reply