Guided Vision in Gemini Live: built for accessibility

Google has introduced Guided Vision in Gemini Live, a feature designed to provide real-time visual assistance through conversational audio descriptions and verbal prompts. Users can share their device’s camera to receive dynamic descriptions of their surroundings and ask follow-up questions in natural language. The tool was developed in collaboration with the blind and low-vision community to ensure practical usability in daily tasks. Compatible Android devices running version 9 or higher can access the feature where Gemini Live is supported.

The feature was trained using tens of thousands of hours of visual data interpreted by Aira, a company specializing in visual assistance for blind and low-vision individuals. Over 1,000 members of Aira’s Trusted Tester network participated in refining the model, while Aira specialists worked with Google’s teams to establish safety guardrails. Guided Vision is intended to assist with everyday tasks such as locating objects, preparing meals, or reading fine print in low lighting. It is not a replacement for established mobility aids or safe-travel practices.

Guided Vision operates within a Gemini Live session, where users receive proactive verbal cues to adjust their camera angle if it is misaligned. For example, the system may prompt users to pan slowly, tilt downward, or step back to capture clearer visual context. The feature is designed to be intuitive, allowing users to explore their environment through spoken guidance without needing to interpret visual information themselves.

Google emphasizes that Guided Vision is an assistive utility and may occasionally make mistakes, as is common with generative AI technologies. The feature is available in regions and languages where Gemini Live is supported, and users must ensure their Gemini app and Android system software are updated to the latest versions. Google also notes that the tool is not intended for navigation, safe-travel guidance, or obstacle detection.