Google launches Guided Vision for Gemini Live: What is it and how it works

Google has announced that it is rolling out Guided Vision in Gemini Live for all compatible Android smartphones, introducing a voice-forward, real-time visual interpretation tool engineered for individuals who are blind, have low vision, or need intuitive visual help in daily life. Developed in collaboration with the accessibility community and previewed earlier this month alongside broader Android accessibility upgrades, the feature converts Gemini into an active conversational partner capable of narrating surroundings, answering follow-up queries and offering spoken camera-framing directions.Built in partnership with AiraTo prepare Gemini Live for unpredictable, real-world conditions, Google partnered with Aira, analysing tens of thousands of hours of visual interpretation data. According to the company, over 1,000 participants from Aira’s Trusted Tester network tested and refined the system during everyday tasks. Aira specialists collaborated directly with engineering teams as domain experts to help formulate, assess, and stress-test the model’s safety guardrails.Conversational camera reframingA key element of Guided Vision is its proactive framing assistance. When a user streams their phone camera during a Gemini Live interaction, the assistant analyses the view in real time. If the camera is positioned too close, tilted too high, or off-center, Gemini provides spoken guidance, such as asking the user to pan slowly to the right, tilt downward, or step back, ensuring it gets the visual clarity needed to provide precise answers.Guided Vision for Gemini Live: Everyday use casesThe tool is engineered for quick everyday visual verification across multiple scenarios. For example, it can read nutritional facts on grocery packages, deciphers dials on appliances and washing machines, or interprets menus in low-light restaurant environments.It scans immediate spaces to find lost items, such as a dropped earbud on a rug or a specific spice container inside a crowded pantry. Identifies shades, prints and cuts, helping users determine if clothing items match or confirming the colour of a specific coat.It can also give descriptive walkthroughs of unfamiliar layouts, rooms and items arranged across tabletops, and supports spoken conversations across multiple languages, incorporating feedback from blind and low-vision communities across India, Brazil, Singapore, Indonesia and Japan.Beyond assisting blind and low-vision users, the feature offers practical utility for older adults, individuals with lower literacy levels, or anyone attempting to decipher text in poor lighting.How to setup and activate on AndroidGuided Vision is designed to launch quickly through three distinct routes on Android.Gemini App Settings: In the Gemini mobile app, navigate to profile settings, turn on "Use Guided Vision in Live," open Gemini Live, and tap to share the camera.System Accessibility Shortcuts: Go to Android Settings > Accessibility > Vision assistance > Guided Vision to assign activation via the floating accessibility button, a two-finger swipe, or a simultaneous press of both volume keys.TalkBack Integration: Users utilizing Google TalkBack can trigger the TalkBack menu with a three-finger tap and select Guided Vision directly.Google noted that Guided Vision is an assistive utility powered by generative AI and may make mistakes.You use AI every day. Now get your AI Quotient. Take the AIQ test.