Why we reject findings that fail the exact character offset test

There is a design decision near the bottom of every legal AI product, and it decides how much the output is worth. The question is what a finding is allowed to point at. Point it at a summary of a chunk and the finding is a claim about a paraphrase. Point it at a character range in the document the lawyer uploaded and the finding is a claim about the contract.

The failure mode, concretely

Take an indemnities review. The retrieval layer splits a 34-page MSA into chunks, summarises each one, and the model reasons over the summaries. A summary of clause 11 reads "supplier indemnifies customer for third-party IP claims, subject to the liability cap". That is a fair summary. It is also missing the carve-out two sentences later that lifts IP claims out of the cap entirely.

The model now produces a finding — "IP indemnity is capped" — that is confidently wrong, and worse, it is unfalsifiable from the interface. The reader cannot tell whether the model misread the clause or the summariser dropped the sentence, because the interface never shows them the clause. They see a claim, a confidence score, and a page number that is close enough to look right.

What we do instead

An agent that proposes a finding must attach the clause it relies on, verbatim. Before the finding is allowed to surface, CitationGate locates that quote in the source text and returns real character offsets. The rules are deliberately blunt:

• A finding with no citation is rejected outright — not downgraded, not shown with a warning.
• The first match attempt is whitespace-flexible exact; a fuzzy word-shingle fallback covers long spans on first verification only. Either way the result carries offsets and a match type (exact or fuzzy).
• Quotes under 10 characters are rejected. "the cap" matches everywhere and proves nothing.
• A finding survives only if at least one of its citations verifies.
• Every decision — verified, failed, passed, rejected — writes an audit entry.

Re-verification is stricter than verification

This is the part that surprises people. When a document changes, we re-run the citations in exact-only mode: no fuzzy fallback. If someone edits an amount, a defined term or a date, the finding that cited the old text fails to match and is marked stale rather than quietly re-anchored to whatever text is nearest.

A fuzzy re-match would be the friendlier behaviour and the wrong one. "Liability is capped at ₦50,000,000" and "Liability is capped at ₦500,000,000" are one character apart and shingle almost identically. Any matcher tolerant enough to survive ordinary redrafting is tolerant enough to move a finding onto a number that changed the deal.

The one exception we make, and why we say so out loud

Some of the most valuable findings are about what a contract does not say — a missing indemnity, an absent liability cap, no governing-law clause. These have no quote to gate, structurally. They carry a non-quote evidence type instead ("the playbook requires this and extraction found no matching provision"), they bypass the quote gate by design, and they default to high supervision, because a false "this is missing" claim is expensive in a way a false "this is present" claim usually is not.

That exception is documented rather than hidden. A verification architecture with an undisclosed exception is not a verification architecture — it is a marketing claim with a gap in it.

What it costs

Gating on verbatim spans throws away findings. Some of them were probably true. We take that trade because the alternative is a review a partner cannot sign: an unverifiable finding does not save a lawyer time, it moves the work from reading the contract to auditing the tool.