Resource · Guide
How to check what an AI tells you.
Nobody develops a sense for when AI is right. What works is checking, and the checks below are ordered by what they cost you in minutes, cheapest first, because the whole point is making verification fast enough that it stops being the bottleneck.
Why this matters more in 2026
Error rates have fallen, and the errors that remain got harder to see. The gibberish era is over: today’s mistakes are plausible, specific, and delivered in exactly the same confident tone as the truths around them, because models are trained in ways that reward confident guessing. A journalism study found AI search tools misattributed sources in over 60 percent of tests, and fabricated references in published academic papers have multiplied tenfold since 2023. Confidence of tone is zero evidence of truth.
The checks
1. Check that sources exist, then check they say it
~30 sec per citation
Click every load-bearing link. Paste citations into Google Scholar or the library catalog. And read one level deeper, because the dominant failure has shifted: outright fake citations are rarer now, while real sources that don't say what the AI claims are common. Librarians call these ghost citations, real authors and a plausible title mixed together. Existence is half the check; fidelity is the other half.
2. Pull out the checkable atoms
1 to 2 min
Names, numbers, dates, quotes, titles. Search each one in quotes. A piece of AI writing usually rests on two or three load-bearing facts, and those are what you verify. This is the lateral-reading method university libraries teach, adapted for a source that has no author or publisher to evaluate.
3. Spot-check the part you already know
~1 min
If the output is wrong where you're the expert, distrust it where you're not. One caveat the research insists on: a pass calibrates trust for this kind of task only. A model that nails your syllabus summary has proven nothing about its citations.
4. Give it permission to say “I don't know”
~15 sec, in the prompt
Models guess because their training rewards a confident answer over a blank one; OpenAI's own researchers describe them as test-takers who never leave a question unanswered. Your prompt is one of the few places that incentive reverses. Add “if you're not sure, say so” and, for documents you've uploaded, “only make claims you can support with a direct quote.” If it can't produce the quote, the claim retracts itself.
5. Verify outside the chat
free
Asking the AI “are you sure?” just produces another generation. Fact-checking organizations are unanimous on this. Verification means leaving the conversation: a search, a catalog, a colleague, the primary document.
6. Match the depth to the stakes and the task
scales with stakes
A summary of a document you supplied runs a low error rate; unverified research citations run a high one. An email draft needs a skim; anything with your name going outside the university needs the full treatment. Spend your checking minutes where the failure modes live.
Where the errors live, by task
| Task | Risk | The check |
|---|---|---|
| Summarizing a document you provided | Lowest | Scan for anything not in the original; confirm numbers and names transcribed exactly. |
| Research and literature questions | Highest | Every citation checked for existence and fidelity. Prefer search-grounded modes, and assume long AI research reports contain some dead or invented links. |
| Data analysis | Subtle | Recompute one figure by hand, verify the row counts, ask it to show its work. Never accept a derived statistic without one independent recomputation. |
| Drafting and writing | Low, with a catch | Check any fact, quote, or citation the model added on its own. Models decorate drafts with plausible specifics. |
Know which mode you’re in
The single biggest factor in error rates is whether the answer is grounded: built from documents you supplied or a live search, versus recalled from training memory. Grounding cuts errors by an order of magnitude; no phrasing trick comes close. So upload the document, use search mode for factual questions, and treat citations as receipts that still need checking. (The terms are in the primer.)
Sources
- Tow Center for Digital Journalism, “AI Search Has a Citation Problem”: 1,600 tests across eight AI search engines.
- OpenAI, “Why Language Models Hallucinate”: why confident guessing is trained-in behavior.
- Anthropic, “Reduce hallucinations”: the permission-to-decline and quote-or-retract techniques.
- University of Maryland Libraries, “Fact-checking AI with Lateral Reading”: the check-the-claims method.
- STAT News on the Lancet-published citation study: fabricated references in published papers, 2023 to 2026.