Verifying AI Output: A Checklist Before You Sign the Report

The AI Verification Problem

You've used AI to help build your chronology. It extracted dates, summarized findings, and even flagged inconsistencies. It saved you hours.

Now you're about to sign your report. But before you do, you need to answer one question:

Is the AI output accurate?

This post provides a checklist for verifying AI output before you sign the report — because if the AI is wrong, you're responsible.

Why Verification Matters

1. AI Makes Mistakes

AI is not infallible. It can:

• Extract the wrong dates
• Misinterpret findings
• Miss important details
• Hallucinate (invent information that doesn't exist)

2. You're Responsible

When you sign the report, you're attesting that the content is accurate. If the AI made a mistake, it's your mistake.

3. Opposing Counsel Will Check

If you rely on AI output without verification, opposing counsel will find the errors. And they'll use them to undermine your credibility.

The Verification Checklist

Check 1: Date Accuracy

What to verify:

• Are all dates extracted correctly?
• Are dates normalized to a consistent format?
• Are ambiguous dates resolved correctly?

How to verify:

• Sample 10-20% of dates
• Check against the original records
• Confirm format consistency

Example:

• AI output: "2024-03-04"
• Original record: "03/04/2024" (March 4, 2024)
• Verification: Correct

Check 2: Finding Accuracy

What to verify:

• Are findings summarized accurately?
• Are key details preserved?
• Are findings attributed to the correct provider?

How to verify:

• Sample 10-20% of findings
• Check against the original records
• Confirm provider attribution

Example:

• AI output: "MRI shows disc herniation at L4-5"
• Original record: "MRI lumbar spine: Left paracentral disc herniation at L4-5, impinging on the left L5 nerve root"
• Verification: Accurate summary, but laterality (left) was omitted. Add laterality.

Check 3: Citation Accuracy

What to verify:

• Are page citations correct?
• Do citations point to the right page?
• Are citations to the authoritative version (not duplicates)?

How to verify:

• Sample 10-20% of citations
• Go to the cited page
• Confirm the finding is there

Example:

• AI output: "Pain 8/10 (p. 45)"
• Original record: Page 45 shows "Pain 6/10"
• Verification: Citation is wrong. Correct to "Pain 6/10 (p. 45)" or find the correct page.

Check 4: Completeness

What to verify:

• Are all relevant records included?
• Are all key findings captured?
• Are gaps and missing records identified?

How to verify:

• Compare AI output to your provider inventory
• Check that all providers are represented
• Check that key findings are captured

Example:

• AI output: Chronology includes ER, family doctor, and physiotherapy
• Your inventory: Also includes chiropractor and pain specialist
• Verification: AI missed two providers. Add them.

Check 5: Inconsistency Flags

What to verify:

• Are inconsistencies flagged correctly?
• Are they interpreted correctly?
• Are false positives (flagged but not actually inconsistent) identified?

How to verify:

• Review each flagged inconsistency
• Check against the original records
• Confirm interpretation

Example:

• AI output: "Inconsistency: Pain 8/10 vs. moving comfortably"
• Original records: Pain 8/10 reported at 9 AM, physiotherapy note at 2 PM says "moving comfortably"
• Verification: Inconsistency is real, but timing matters. Add context: "Pain 8/10 at 9 AM, but moving comfortably at 2 PM (after medication)."

Check 6: Causation Opinions

What to verify:

• Did AI offer causation opinions? (It shouldn't)
• If it did, remove them
• Causation opinions are yours, not AI's

How to verify:

• Scan AI output for causation language (e.g., "caused by," "related to")
• Remove or revise

Example:

• AI output: "The disc herniation was caused by the MVA"
• Verification: AI should not offer causation opinions. Remove or revise to: "The disc herniation was diagnosed after the MVA. Causation is a clinical opinion."

Check 7: Missing Context

What to verify:

• Did AI miss context that changes the interpretation?
• Are there explanations for gaps, inconsistencies, or findings?

How to verify:

• Review AI output for gaps and inconsistencies
• Check original records for explanations
• Add context

Example:

• AI output: "Gap in treatment from 02/10 to 02/28"
• Original records: Counsel letter notes plaintiff was on vacation
• Verification: Add context: "Gap in treatment from 02/10 to 02/28 (plaintiff on vacation, per counsel letter)."

Check 8: Hallucinations

What to verify:

• Did AI invent information that doesn't exist in the records?
• Are there findings that you can't verify?

How to verify:

• If a finding seems surprising, check the original record
• If you can't find it, it may be a hallucination
• Remove unverifiable content

Example:

• AI output: "Plaintiff reported prior low back pain in 2018"
• Original records: No mention of 2018 low back pain
• Verification: This may be a hallucination. Remove or verify.

The Verification Workflow

Step 1: Sample and Verify (30-60 minutes)

• Sample 10-20% of dates, findings, and citations
• Verify against original records
• Correct errors

Step 2: Check Completeness (15-30 minutes)

• Compare AI output to provider inventory
• Ensure all providers and key findings are included

Step 3: Review Inconsistency Flags (15-30 minutes)

• Review each flagged inconsistency
• Add context and interpretation

Step 4: Remove AI Causation Opinions (5-10 minutes)

• Scan for causation language
• Remove or revise

Step 5: Add Missing Context (15-30 minutes)

• Review gaps and inconsistencies
• Add explanations from the record

Step 6: Check for Hallucinations (15-30 minutes)

• Review surprising findings
• Verify against original records
• Remove unverifiable content

What This Looks Like in Practice

Before verification:

Problems:

• Pain was actually 6/10 (citation wrong)
• ROM improved, but pain not mentioned (finding incomplete)
• Returned to work, but restrictions not mentioned (context missing)

After verification:

Better: Verified, corrected, and complete.

Key Takeaways

1. AI makes mistakes. You're responsible for catching them.
2. Verify date accuracy, finding accuracy, citation accuracy, completeness, inconsistency flags, causation opinions, missing context, and hallucinations.
3. Sample 10-20% of entries and verify against original records.
4. Correct errors, add context, and remove unverifiable content.
5. Don't let AI offer causation opinions — those are yours.
6. Build verification into your workflow. It takes 1-2 hours and prevents liability.

What's Next?

In our next post, we'll tackle "What Happens When Opposing Counsel Challenges Your Chronology" — how to defend your work under cross-examination.

This post is part of our series on defensible medical record review. For more, see our 60 AI Prompts for IME Physicians Reviewing Medical Records.

Questions for Readers:

• Do you use AI in your record review? What's your verification process?
• Have you caught AI errors before they became problems?
• What's your approach to AI-generated causation opinions?