Google Research shows AMIE medical AI handling simulated video consultations

Google Research and Google DeepMind have advanced AMIE, the company's research medical AI system, from text-based clinical dialogue into real-time audio-visual consultations. The August 11 announcement describes AMIE (Video) as a prototype built on Gemini and Project Astra that can speak with a patient actor, observe video and audio cues, guide virtual physical examination steps and update diagnostic reasoning during the conversation.

The work matters because telehealth is not only a chat transcript. Clinicians often rely on gait, visible discomfort, breathing sounds, facial cues and how a patient performs a simple examination maneuver. Google's research post argues that text-only medical AI misses much of that signal, forcing patients to translate physical symptoms into written descriptions and potentially making the interaction harder for people with limited digital or health literacy.

How the system was tested

Google says AMIE (Video) uses an asynchronous multi-agent architecture. A patient-facing talker agent keeps the spoken consultation moving, a planner agent refines clinical reasoning in the background, and a perception agent continuously reviews the audio and video stream for medically relevant cues. The company says this split is intended to preserve natural conversational latency while still allowing deeper diagnostic reasoning.

In the reported randomized study, Google evaluated 100 clinical scenarios across 300 live consultations. The comparison included AMIE's video configuration, a text-only AMIE baseline and consultations by board-certified primary care physicians through the same video interface. Independent physician evaluators reviewed the sessions using clinical rubrics, while trained patient actors also rated the interaction experience.

• Clinical evaluators rated AMIE (Video) on par with primary care physicians across core competencies including history-taking, diagnostic accuracy, management appropriateness and communication quality, according to Google.
• Google says the video system performed better than the text baseline on several audio-visual and virtual examination tasks.
• Patient actors preferred the video experience over text chat and rated the system favorably for empathy, rapport and confidence in care.

The limitations are just as important as the headline result. Google says the study used professional patient actors in simulated encounters, not real patients presenting their own medical conditions. The scenarios also omitted conditions that could not be realistically portrayed by actors, and the company notes that targeted evaluations still found occasional perceptual and reasoning errors.

That makes AMIE (Video) a research milestone rather than a deployable clinical product. Google says real-patient validation, broader clinical coverage and stronger safety frameworks would be needed before drawing conclusions about real-world utility. Still, the demonstration points toward a next phase of health AI in which systems are judged not only on written answers, but on whether they can participate in the sensory, interactive flow of a clinical visit.

Sources

• blog.google
• research.google

Cover photo by Tima Miroshnichenko on Pexels, used under the Pexels License.