Microsoft\u2019s AI can now read your screen \u2014 or rather, the websites you\u2019re browsing.<\/p>\n\n\n\n
On Thursday, the company began rolling out a limited, U.S.-only preview of Copilot Vision, a tool that can understand and respond to questions about sites you\u2019re visiting using Microsoft Edge. Gated behind Copilot Labs, an opt-in program for experimental AI capabilities, Copilot Vision can analyze text and images on web pages to answer queries like \u201cWhat\u2019s the recipe for this lasagna?\u201d<\/p>\n\n\n\n
Note that Copilot Labs requires a subscription to Microsoft\u2019s Copilot Pro plan, which costs $20 per month.<\/p>\n\n\n\n
Beyond answering questions, Copilot Vision can summarize and translate text, and handle tasks like spotlighting discounted products in a store catalog. It can also serve as a game assistant, for example offering pointers during matches on Chess.com.<\/p>\n\n\n\n
\u201cWhen you choose to enable Copilot Vision, it sees the page you\u2019re on, it reads along with you, and you can talk through the problem you\u2019re facing together,\u201d Microsoft wrote in the blog post shared with TechCrunch. \u201cIt\u2019s a new way to invite AI along with you as you navigate the web, tucked neatly into the bottom of your Edge browser whenever you want to ask for help.\u201d<\/p>\n\n\n\n
No doubt eager to avoid more bad press from AI privacy fumbles, Microsoft is stressing that Copilot Vision deletes data following every session. Processed audio, images or text aren\u2019t stored or used to train models, the company says \u2014 at least not in this preview release.<\/p>\n\n\n\n
Copilot Vision is also limited in the types of websites it can view. For the time being, Microsoft\u2019s blocking the feature from working on paywalled and \u201csensitive\u201d content, limiting Vision to a pre-approved list of \u201cpopular\u201d sites.<\/p>\n","protected":false},"excerpt":{"rendered":"
Microsoft\u2019s AI can now read your screen \u2014 or rather, the websites you\u2019re browsing. On Thursday, the company began rolling out a limited, U.S.-only preview of Copilot Vision, a tool that can understand and respond to questions about sites you\u2019re visiting using Microsoft Edge. Gated behind Copilot Labs, an opt-in program for experimental AI capabilities, […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12],"tags":[],"class_list":["post-1034","post","type-post","status-publish","format-standard","hentry","category-tech"],"_links":{"self":[{"href":"https:\/\/www.timesofmedia.com\/news\/wp-json\/wp\/v2\/posts\/1034","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.timesofmedia.com\/news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.timesofmedia.com\/news\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.timesofmedia.com\/news\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.timesofmedia.com\/news\/wp-json\/wp\/v2\/comments?post=1034"}],"version-history":[{"count":1,"href":"https:\/\/www.timesofmedia.com\/news\/wp-json\/wp\/v2\/posts\/1034\/revisions"}],"predecessor-version":[{"id":1035,"href":"https:\/\/www.timesofmedia.com\/news\/wp-json\/wp\/v2\/posts\/1034\/revisions\/1035"}],"wp:attachment":[{"href":"https:\/\/www.timesofmedia.com\/news\/wp-json\/wp\/v2\/media?parent=1034"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.timesofmedia.com\/news\/wp-json\/wp\/v2\/categories?post=1034"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.timesofmedia.com\/news\/wp-json\/wp\/v2\/tags?post=1034"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}