We publish how the numbers were made.
The 92.5% and 96.5% on our product pages come from two internal measurements, July 2026. This page shows what was measured, how, what the pipeline missed — enumerated one by one — and where the measurement itself falls short.
- 92.5%
- requirement-extraction recall — full golden set, 173 items
- 96.5%
- machine-certain precision — after adjudicating every flagged case
- 18 / 19
- defects caught in the adversarial judgment benchmark
- 100%
- run-to-run stability of the final pass/block posture (15 packages × 2 runs)
Two real, published negotiated-contract service tenders (8 documents, table-preserving parse). An adjudicator authored the full requirement set — 85 and 88 items, 41 of them critical — while barred from seeing any pipeline output, so the answer key could not be shaped by what the pipeline happens to extract.
Every item carries a verbatim anchor into the source text, and all 173 anchors pass machine verification. Table rows are included as items — tables are where Korean tender documents put the substance.
A single extraction pass scores 82.1% recall against the golden set. Adding one critic pass — a second call that only hunts for requirements the first pass missed — lifts it to 92.5%, beating an equal-cost union of two independent passes (88.4%). That is the configuration we ship: one extra LLM call per document.
The critic over-extracts by design; the deterministic grounding layer absorbs it — every added item, grounded rate 100%.
Real bid submissions are exempt from disclosure in Korea, so they cannot be obtained. Instead: for three real published tenders (engineering design, solar installation works, security services — HWP and PDF), we generated fully compliant bid packages, then injected 45 defects with seeded, fixed mutations — so every defect label is certain — plus 15 harmless-rewording traps.
Machine-certain precision: 96.5% after adjudicating every flagged case. The deterministic layer caught every missing document, every identifier tamper, and every total-amount tamper — without LLM involvement.
Every miss was adjudicated individually — none are mysteries. Of 45 injected defects, the judgment layer missed 9:
- 3 — the notice figure lived in a table the (then-current) parser dropped; the table-preserving parse path since became the default
- 2 — the extracted requirement summary omitted the number needed to catch the deviation (granularity, not judgment)
- 2 — genuinely lenient judgment calls: a deleted section still voted “covered”. These are the real judgment failures: 2 of 45.
- 1 — an upstream extraction miss propagated downstream
- 1 — label turned out invalid: the deleted content was covered by another document
Also disclosed: 2 of the 15 harmless-rewording traps drew a “partial” flag (false positives), and the extraction residue clusters in edge cases such as authoring instructions inside a proposal’s table-of-contents table.
All 15 packages were verified twice, independently. The final pass/block posture agreed 15/15. What flickers between runs is confined to borderline grades — “is this commitment strong enough” — never the defect-anchored findings (missing documents, identifier and total mismatches, dropped requirements).
- The bid packages are synthetic. Real vendors’ proposals differ in style, structure, and error distribution — figures here are benchmark performance, not a guarantee on your documents.
- Judgment-of-merit factors — evaluation scoring, how track-record rules are interpreted — are out of scope; only document, figure, and commitment-level checks were measured.
- With 45 defect labels, one or two percentage points around 95% precision are statistically indistinguishable.
- Scoring and miss adjudication were performed by an LLM adjudicator, not a human domain expert.
The reports the pipeline produces are public too — read one before sending anything.