What's new
Explainer14 Nov 20265 min read

A citation can point to a real passage and still fail the claim

In our invented audit, 95 of 100 citation pointers resolve but only eighty passages support their claims, so the passage has to be checked against the whole sentence before an answer counts as grounded.

Following a citation lets a reviewer inspect the material behind an answer, which is a good start. It does not say whether that material establishes what the answer claims. A real passage about a company does not support every statement the model might make about that company.

The Claude citations guide, read on 14 November 2026, describes source locations and returned cited text. Those fields help find the evidence. The audit we propose checks whether each location is valid and whether the passage supports the claim, as two separate questions, so a citation being present is never treated as acceptance.

The unit is a claim

An answer is split into statements that can be checked, with the original sentence kept beside each split so qualifications survive. A sentence saying a feature is available in a particular region on a particular plan needs evidence for those conditions, not merely for the feature's existence.

For the teaching example, assume 100 claims with one citation each. Ninety-five pointers resolve to the intended source, and eighty of those passages support the complete claim under the review rule, meaning the standard the reviewer applies to decide that a passage supports a claim.

Constructed counts for 100 pairs, one cited passage per claim. Supported claims are a subset of valid pointers.
Audit gate in a synthetic setClaim-citation pairsShare of all pairs
Valid source pointer9595%
Fully supported claim8080%

Pointer validity is 95%. Support across the original set is 80%, and among valid pointers it is about 84.2%. Fifteen claims point somewhere real without being supported. Keeping the original denominator stops a citation-format success from hiding an evidence failure.

Conditions around the passage

The subject, date, unit and scope all need checking. A number about one product tier should not become a number about the entire service, and a planned change should not become a shipped feature. The relevant qualification should be saved alongside the passage when the claim is reviewed.

A source reporting its own result is also different from an independent finding. A citation can establish accurately that a lab made a claim while the claimed performance stays unverified, and the answer should be worded to match that narrower evidence.

Where several sources are required, completeness has to be defined before scoring. A comparison may need one passage for each side, and counting a claim as supported because one of its citations resolves would skip the second half.

Pointers go stale with the source version

A pointer needs the document version it refers to. A character offset in revised text may land on a different sentence, and a page reference can shift after a new cover sheet is added. A stable source identifier plus a version or retrieval date, in whatever form suits your system, has to be kept with the pointer, and the matching version of the source has to be kept or recoverable, because the identifier and date alone cannot bring back a version that is no longer available.

When the viewer is built, it should follow the API's documented indexing conventions and show the cited material exactly as returned or as recovered from the matching source. Repairing an invalid location by picking a plausible passage, then counting the original pointer as valid, inflates the score.

For public reports, publish only material you are allowed to quote and retain. A source link, a brief permitted excerpt and the review outcome can be enough. Private documents and account-specific records should not become public fixtures for convenience.

Failures that suggest a repair

A missing pointer, a wrong location, a passage that only partly supports the claim and an outdated source call for different repairs, so they are labelled separately. A model change may not fix a viewer that opens the wrong document version.

Questions whose answer is absent from the supplied documents belong in the set. There, the right behaviour may be to say the material does not establish the answer, and an unsupported answer with a decorative citation fails the same review rule as one with no citation.

Your own results log needs the claim count, the pointer-valid count, the fully supported count and the review rule.