"Check every page of this appraisal." "Confirm no contract in this data room has a change-of-control clause." These aren't retrieval questions — they need coverage, and a tool optimised for finding the best evidence can't tell you whether it checked everything.
An exhaustive review of all 9,000 documents in a data room for a change-of-control clause: one child run per document, a reduce run over all of them, and a coverage number — 9,000 of 9,000 processed — instead of a confidence score.
A partner asks whether any of the 9,000 documents in the data room contains a change-of-control clause. The honest answer, with a retrieval-based tool, is "probably" — and probably isn't good enough to sign off on.
Top-k is built to answer "what's the best evidence for this question", which is exactly the wrong optimisation for "did we check everything".
Absence of evidence is not evidence of absence — but a tool that only surfaces the most relevant passages can't tell the difference between the two, and a reviewer relying on it doesn't know which one they got.
More relevant chunks is still relevance-ranked, not exhaustive. The 51st most relevant chunk might be the one with the clause, and nothing changes about whether it gets seen.
It's the only reliably exhaustive option today, and it costs days of billable time for a question a machine can check in the background — while the deadline doesn't move.
Clauses that matter are rarely phrased with the exact term a keyword search expects, and a synonym or a differently structured provision slips straight past a literal match.
A clean-looking summary from a top-k system reports what it found, not what it checked. There's no number anywhere that says how many of the 9,000 documents it actually looked at.
Most tasks read top-k retrieved chunks, which is correct for "answer this question". A task can instead declare exhaustive coverage, which changes how it executes entirely.
Exhaustive coverage is a map-reduce: every document in scope gets its own run against the task, and a final reduce run aggregates the results into one answer.
The parent run records documents_in_scope, documents_processed, and failures — and cost and wall-clock are shown before the run launches, not discovered after.
9,000 of 9,000 documents processed is a fact you can check against the data room's own document count — not a self-reported confidence from the model.
A failed child run is visible and retried, not silently dropped from the aggregate — so "we checked everything" is actually true when the run says so.
9,000 child runs at a known per-run cost is a number you see and approve before it starts, not a surprise on next month's invoice.
The final "no clause found" answer is a run like any other — reviewable, with every child result it aggregated still attached underneath it.
