Verifying Facts, Citations, and Numbers
Practical techniques for checking AI's specific claims against independent sources: extracting claims, reading laterally, confirming citations exist and say what's claimed, and sanity-checking numbers.
Verification Means Leaving the Conversation
The single most important rule of product discernment: you can’t verify AI output from inside the AI conversation. Asking “Are you sure?” tests whether the model is consistent, not whether it’s right. Real verification compares a claim with something independent: an original document, an official source, a dataset, a measurement, or a qualified person.
This lesson gives you a repeatable method.
- 1 Extract
List the checkable claims
- 2 Rank
Riskiest and most important first
- 3 Source
Find an independent, original source
- 4 Compare
Does the source say exactly that?
- 5 Mark
Confirmed, corrected, or removed
Step 1: Extract the Claims
Long, fluent paragraphs hide specifics. Pull them out. You can do this yourself or ask the AI to help:
List every factual claim in the text below as a numbered list.
Include every name, date, number, statistic, quote, citation, and
statement about what a law, policy, or product says or does.
Don't evaluate them, just list them.
[paste text]
Using AI to extract claims is fine because you’ll check the list against the original text. Using AI to verify them is not.
Step 2: Rank by Risk
Not every claim needs equal effort. Start with claims that are:
- Load-bearing: the conclusion depends on them.
- Precise: exact numbers, dates, quotes, and section numbers.
- Surprising: if it contradicts what you expected, check it before you repeat it.
- About recent or niche topics: where training data is thin.
Step 3: Find an Independent Source
Good sources, roughly from strongest to weakest:
| Source type | Examples | Watch out for |
|---|---|---|
| Primary | The law itself, the original study, the company’s filing, the official dataset | Make sure it’s the current version |
| Authoritative secondary | Government agencies, professional bodies, reference works | Check the date |
| Reputable reporting | Established news outlets, trade publications | Reporting can simplify or misread the original |
| Everything else | Blogs, forums, AI-written web pages | May repeat the same error, or be generated by AI themselves |
Use lateral reading: instead of studying one page closely to decide if it’s trustworthy, open new tabs and see what other sources say about the claim and about the source itself. Fact-checkers rely on this because it’s faster and more reliable than judging a page by how professional it looks.
Step 4: Check Citations Twice
A citation can fail in two ways:
- It doesn’t exist. Search for the exact title in a library catalog, a scholarly search engine, or the publisher’s site. Check that the authors, year, and venue match.
- It exists but doesn’t say that. This is more common and harder to spot. AI may cite a real paper for a claim the paper never makes. Open the source and find the passage. If you can’t find support in the text, the claim is unsupported, regardless of how real the citation is.
Common mistake: assuming that AI tools which show sources or links have verified their claims.
Source links are a big improvement, because they give you somewhere to check. But a link only proves the page exists. The AI may summarize it wrongly, pull a figure from the wrong year, or attach a link to a sentence it doesn’t support. Click through and find the actual sentence, especially for anything load-bearing.
Step 5: Sanity-Check Numbers
Before you even look up a number, ask whether it’s plausible:
- Magnitude: Is it in the right ballpark? If AI says a town of 8,000 people has 40,000 schoolchildren, something’s wrong.
- Units and time frames: Per month or per year? Thousands or millions? Percent or percentage points?
- Internal consistency: Do the parts add up to the total? Do percentages sum to roughly 100 where they should?
- Base rates: “Sales grew 300%” from what? A tiny base makes a dramatic percentage.
- Precision: A suspiciously exact figure like “42.7% of managers” with no source is a hallucination red flag.
Then trace it to a source like any other claim.
- Pick a topic where public facts are easy to check, such as a local landmark, a historical event, or a public company. Ask an AI assistant:
Write a 200-word briefing on [topic] for a new employee, with at least five specific facts (dates, numbers, names) and two sources. - Extract: List every specific claim, either yourself or using the extraction prompt above.
- Rank: Star the three that matter most.
- Verify: For each starred claim and both sources, find an independent source. For sources, confirm they exist and support the specific claim.
- Mark each claim: ✅ confirmed, ✏️ corrected, or 🗑️ removed. Note how long the whole check took.
- Reflect: if this briefing had gone out unchecked, what would the worst error have been?
1. An AI cites a real, well-known study to support a claim. What's still left to check?
Citations fail in two ways: they don't exist, or they exist but don't support the claim. You have to read the relevant part of the source.
2. What is lateral reading?
Lateral reading means leaving the page to see what others say. It's faster and more reliable than judging a source by its appearance.
3. An AI summary says a 12-person nonprofit 'serves 2.4 million clients a year.' What's your first move?
A magnitude sanity check catches this before you even search. It may be a wrong unit, a wrong organization, or an invented figure.
Up Next
Accurate facts can still add up to a lopsided picture. The next lesson looks at bias and the perspectives AI output leaves out.