Key points

  • On 30 shared cases, freely navigating models were correct in 30.0% of runs, compared with 77.0% for radiologists.
  • Six models read 100 emergency CT and MRI examinations under four viewing conditions, for 4800 runs in total.
  • Models formed a first hypothesis after a median of 10.8% of slices and kept it in 63.1% of runs.
  • Median stated confidence in final answers was 90%, but only 38.9% of those answers were correct.

AI agents built on vision-language models were much less accurate than radiologists when they read emergency CT and MRI scans. That is the finding of a preprint posted on arXiv, which has not yet been peer-reviewed. On 30 cases that both groups read, models that could browse the images freely got the diagnosis right in 30.0% of runs. Nine radiologists got it right 77.0% of the time.

Vision-language models are AI systems that can handle images and text together. In this retrospective study, six of them read 100 emergency examinations taken from a public archive. Each model tried every case twice under four setups: browsing freely, browsing from a chosen key slice, or getting either all slices or only the key slices in a single request. That added up to 4800 runs. A separate large language model graded the answers, and the team checked that grader against two radiologists.

The models did better when they were handed images up front. Pooled accuracy was 32.6% with free browsing and 37.9% when browsing began from a key slice. It reached 42.2% with all slices and 42.5% with key slices in one request.

The researchers also tracked how the models navigated the images. They formed a first guess after looking at a median of 10.8% of the slices, and they kept that guess in 63.1% of runs. When the first guess was wrong, the final answer was correct only 12.4% of the time. Confidence was also out of line with accuracy: models put their median confidence at 90%, yet only 38.9% of their final answers were correct.

Why it matters

The study suggests that, no matter how they explored the images, current vision-language model agents stayed far below radiologists on emergency imaging. It also found that the models tended to stick with early guesses and seldom fixed wrong ones.

What we don't know yet

This is a retrospective, non-peer-reviewed preprint based on 100 archived cases. The radiologist comparison covered only 30 of those cases, and the answers were graded by a language model rather than by radiologists directly.

Terms in this story

vision-language model
An AI system trained to process both images and text, so it can answer written questions about pictures.
slice
One of the many cross-sectional images that together make up a CT or MRI scan.
retrospective study
Research that looks back at cases or data that already exist instead of following new patients forward.

Sources and verification

  1. Diagnostic Accuracy and Navigation Behavior of Vision-Language Model Agents Reading Emergency CT and MRI Examinations arXiv · Nov 19, 2026 · preprint, not peer-reviewed
Fact-checked against the source on November 20, 2026: 26 factual claims verified.
Written in our own words: longest phrase shared with the source is 12 words.
Related image from Wikimedia Commons (CC BY 4.0), checked for relevance and that it is not AI-generated. It is not an image from the study.

DexScience articles are produced with AI assistance and an automated fact-check against the linked sources. Spotted an error? Tell us and we will correct it openly.