Verifying Facts, Citations, and Numbers
Module 2 · Checking the Product

Verifying Facts, Citations, and Numbers

Practical techniques for checking AI's specific claims against independent sources: extracting claims, reading laterally, confirming citations exist and say what's claimed, and sanity-checking numbers.

Verification Means Leaving the Conversation

The single most important rule of product discernment: you can’t verify AI output from inside the AI conversation. Asking “Are you sure?” tests whether the model is consistent, not whether it’s right. Real verification compares a claim with something independent: an original document, an official source, a dataset, a measurement, or a qualified person.

This lesson gives you a repeatable method.

  1. 1 Extract

    List the checkable claims

  2. 2 Rank

    Riskiest and most important first

  3. 3 Source

    Find an independent, original source

  4. 4 Compare

    Does the source say exactly that?

  5. 5 Mark

    Confirmed, corrected, or removed


Step 1: Extract the Claims

Long, fluent paragraphs hide specifics. Pull them out. You can do this yourself or ask the AI to help:

List every factual claim in the text below as a numbered list.
Include every name, date, number, statistic, quote, citation, and
statement about what a law, policy, or product says or does.
Don't evaluate them, just list them.

[paste text]

Using AI to extract claims is fine because you’ll check the list against the original text. Using AI to verify them is not.

Step 2: Rank by Risk

Not every claim needs equal effort. Start with claims that are:

  • Load-bearing: the conclusion depends on them.
  • Precise: exact numbers, dates, quotes, and section numbers.
  • Surprising: if it contradicts what you expected, check it before you repeat it.
  • About recent or niche topics: where training data is thin.

Step 3: Find an Independent Source

Good sources, roughly from strongest to weakest:

Source typeExamplesWatch out for
PrimaryThe law itself, the original study, the company’s filing, the official datasetMake sure it’s the current version
Authoritative secondaryGovernment agencies, professional bodies, reference worksCheck the date
Reputable reportingEstablished news outlets, trade publicationsReporting can simplify or misread the original
Everything elseBlogs, forums, AI-written web pagesMay repeat the same error, or be generated by AI themselves

Use lateral reading: instead of studying one page closely to decide if it’s trustworthy, open new tabs and see what other sources say about the claim and about the source itself. Fact-checkers rely on this because it’s faster and more reliable than judging a page by how professional it looks.

Step 4: Check Citations Twice

A citation can fail in two ways:

  1. It doesn’t exist. Search for the exact title in a library catalog, a scholarly search engine, or the publisher’s site. Check that the authors, year, and venue match.
  2. It exists but doesn’t say that. This is more common and harder to spot. AI may cite a real paper for a claim the paper never makes. Open the source and find the passage. If you can’t find support in the text, the claim is unsupported, regardless of how real the citation is.
⚠️ Common Pitfall: Trusting Links Because They're Clickable

Common mistake: assuming that AI tools which show sources or links have verified their claims.

Source links are a big improvement, because they give you somewhere to check. But a link only proves the page exists. The AI may summarize it wrongly, pull a figure from the wrong year, or attach a link to a sentence it doesn’t support. Click through and find the actual sentence, especially for anything load-bearing.

Step 5: Sanity-Check Numbers

Before you even look up a number, ask whether it’s plausible:

  • Magnitude: Is it in the right ballpark? If AI says a town of 8,000 people has 40,000 schoolchildren, something’s wrong.
  • Units and time frames: Per month or per year? Thousands or millions? Percent or percentage points?
  • Internal consistency: Do the parts add up to the total? Do percentages sum to roughly 100 where they should?
  • Base rates: “Sales grew 300%” from what? A tiny base makes a dramatic percentage.
  • Precision: A suspiciously exact figure like “42.7% of managers” with no source is a hallucination red flag.

Then trace it to a source like any other claim.


🧪 Try It: Verify a Short AI Briefing
  1. Pick a topic where public facts are easy to check, such as a local landmark, a historical event, or a public company. Ask an AI assistant:
    Write a 200-word briefing on [topic] for a new employee, with at
    least five specific facts (dates, numbers, names) and two sources.
  2. Extract: List every specific claim, either yourself or using the extraction prompt above.
  3. Rank: Star the three that matter most.
  4. Verify: For each starred claim and both sources, find an independent source. For sources, confirm they exist and support the specific claim.
  5. Mark each claim: ✅ confirmed, ✏️ corrected, or 🗑️ removed. Note how long the whole check took.
  6. Reflect: if this briefing had gone out unchecked, what would the worst error have been?
🧠 Knowledge Check

1. An AI cites a real, well-known study to support a claim. What's still left to check?

2. What is lateral reading?

3. An AI summary says a 12-person nonprofit 'serves 2.4 million clients a year.' What's your first move?


Up Next

Accurate facts can still add up to a lopsided picture. The next lesson looks at bias and the perspectives AI output leaves out.