Gemini can now see your screen and judge your tabs

Google's Gemini Live is finally getting the gift of sight. The tech giant has quietly begun rolling out features that transform your humble smartphone into an all-seeing eye for its AI assistant.

The new abilities were uncovered by a Reddit user who later shared a video of the features in action. The upgrade lets Gemini peer through your screen or camera lens and process what it sees. The rollout marks the debut of Google's much-discussed and much-anticipated Project Astra.

Based on the video, Gemini's 'eyes' can analyze your screen in real-time through a "Share screen with Live" button. Gemini has long been able to digest static screenshots, but the update maintains a continuous gaze on your screen, looking at whatever you are on your phone for better or for worse.

The other tool makes your phone's camera Gemini's eye. Google has demonstrated that the AI can precisely discern colors and objects. Whether the final product matches the platonic ideal of the demos isn't clear just yet.

Astra eyes

The new feature is arriving first for Gemini Advanced subscribers paying $20 a month for the Google One plan with extra AI. The rollout is notably democratic in where the feature appears, though, judging from the Xiaomi phone shown by the Reddit user. Google had previously hinted that Pixel and Galaxy S25 owners would have faster or better access to Project Astra.

Other AI assistants with similar seeing tools exist, but they are mostly tied to third-party apps like Microsoft Copilot, ChatGPT, Grok, and even Hugging Face's new HuggingSnap app. Having a real-time screen and camera-connect AI built into Android would certainly help entice users interested in an AI assistant to at least try Gemini.

And Google's timing in releasing the feature is notable as it tries to carve out a lead among AI assistants. Though Amazon has been hyping its new “Alexa Plus” update, it has yet to arrive.

Meanwhile, Apple's upgraded Siri has been delayed multiple times. That leaves Google with a temporary but very real lead in the AI assistant race. Gemini, for all its early hiccups and rebranding drama (RIP Bard), is now doing things that neither Alexa nor Siri can match for the moment.

Google has promised that Project Astra will be the "next-generation assistant" everyone wants to use all day. So keep your (and Gemini's) eyes peeled for new features to arrive in the weeks ahead.

You might also like

How It works

Search Crack for

Latest IT News

Apr 20
Rumored fixes for Windows 11's Start menu sound great, but some folks remain seriously skeptical.
Apr 18
I caved to YouTube Premium's free trial and now I see what the hype is all about, but I can't get past its poor queuing system.
Apr 18
Is 2027 set to be the year of Linux? There are signs that this might actually be the case, for sure.
Apr 17
YouTube finally has an option for users that will allow them to effectively remove Shorts via a zero-minute timer.
Apr 17
Gucci could become the first mainstream luxury brand to launch smart specs, as the company is targeting 2027 for a pair of Android XR glasses.
Apr 17
The Meta Quest 3 and Quest 3S are getting more expensive as Meta blames the RAM crisis.
Apr 16
This half-price YouTube Premium discount with Google One Premium is not to be skipped over.

Latest cracks