The 3-Minute Chart Review Test: AI Clinical Summaries Boosted Comprehension Accuracy from 60% to 83.3%

This article is based on: SBIR Phase I: A tool to automate a narrative patient summary of the medical chart for outpatient physician.

Outpatient clinicians routinely inherit fragmented records spanning multiple sites of care. In practice, “chart review” often becomes a time-boxed exercise: skim fast, make judgment calls about what matters, and hope you didn’t miss the one detail that changes the whole visit. The constraint is simple: most physicians don’t have the luxury of deep pre-visit review, even when the record is huge and clinically important. They typically only have 3 minutes.

So we tested a very practical question: If you give a physician a strict 3-minute review window, do our AI clinical summaries improve what they can accurately extract from the chart?

At a glance

• Research study: NSF SBIR Phase I evaluation. We designed a timed, controlled assessment to measure what clinicians could correctly learn from a chart under realistic time pressure.
• Participants: 2 board-certified physicians
• Cases: 12 distinct patients. The cases were split into two sets of six patients per scenario.
• Assessment items: 60 patient-specific true/false questions (5 per patient). The questions were written in advance by two medically trained researchers familiar with the cases, and the physicians doing the timed review did not have prior knowledge of the questions.
• Comparison:
  • Control: Outside medical record only
  • Intervention: Outside medical record + our AI clinical summary
• Time constraints:
  • 3 minutes to review the chart (per scenario)
  • 2 minutes to answer the 5 true/false questions per patient
• Primary outcome: Percent of questions answered correctly

Results

We had the two physicians review 12 patient charts from cases they had recently treated. In one scenario, they reviewed the outside medical record alone. In the other, they reviewed the outside record plus our AI-generated clinical summary. After each timed review window, they answered five true/false questions per patient.

Across the 60 true/false questions (pooled across both physicians):

• With AI-generated summaries: 83.3% correct
• Without AI-generated summaries (outside record only): 60.0% correct

That’s a 39% relative improvement in comprehension accuracy under the exact same time constraint. Put differently: wrong answers dropped from 40.0% to 16.7%. The AI summary didn’t just make review feel easier, it measurably reduced how much incorrect understanding survived the three-minute chart review.

Why this matters

A tool that increases “what I correctly know about this patient” under time pressure is not just a convenience feature, it changes the quality of the clinician’s starting point. And the starting point is everything. In practice, that can influence:

• whether key history is recognized in time,
• whether follow-ups are noticed,
• whether downstream questions are better targeted.

This is the core takeaway from our NSF Phase I research: when time is fixed and the chart is messy, narrative AI clinical summaries can materially upgrade the first few minutes of patient understanding, well beyond what “record access” alone typically delivers.