How accurate is LegiMatch Brief Check? A blind test on real court documents

Short answer: in our two latest blind tests on 200 real court decisions nobody had used before (5,185 citations), LegiMatch Brief Check caught 1,397 of 1,417 planted case-citation mistakes (98.6%) - wrong year, court, pin cite and invented case 100% each - and called none of the 5,185 real citations nonexistent. With its AI statute check on, it caught 31 of 35 statute sections changed to another real section and 18 of 19 changed to a section that doesn't exist. Every figure below comes from documents nobody had used before.

Disclosure: we make LegiMatch. This is our own test, run on our own servers; the method is below so anyone can repeat it.

Why we publish a rate for each kind of mistake

A citation can be wrong in many ways: the case doesn't exist, the volume is off by a transposed digit, the year or court is wrong, the pin cite is outside the decision, a party's name is misspelled, a quotation has one word changed, or a statute section number points to the wrong provision. A single "accuracy" figure hides which of these a tool misses. Lawyers deserve to know the rate for each, and how often the tool raises a false alarm on a correct citation.

How the test works

1. Fresh documents every run. Each run takes documents never used before: 100 random published court decisions from our library, or 30-50 briefs filed in the U.S. Supreme Court (from supremecourt.gov).
2. Mistakes planted at random. A program changes citations the way real mistakes happen - a wrong year or page, a misspelled party, a changed word in a quotation, a citation to a case that doesn't exist, two citations swapped, a quotation moved to another cited case, a statute section number changed - choosing the citation and the change at random.
3. Counted only when reported correctly. A mistake counts as caught only when the report flags that citation with the right kind of warning. A mistake planted on a quotation the report had already flagged is left out of the count. A quotation moved to another case that really contains the same words (courts repeat each other's phrases) is not counted as a mistake.
4. False alarms measured on the same documents, unaltered. Before anything is planted, each document is checked as filed. Any flag on it is either a real mistake in the document or a false alarm; we count both and check samples by hand.

Results: planted case-citation mistakes

The two latest runs, 20 October 2026 (blind30 and blind31): 200 court decisions, 5,185 citations, every fix in place. Earlier the same day, on 199 other fresh decisions (blind27 and blind28), it caught 98.1% (1,315 / 1,341).

Results: statutes

Two runs, 20 October 2026, with the AI meaning check on: 100 court decisions (blind29) and 30 Supreme Court briefs (briefs_scotus12). Of the 4 missed swaps, two sat in a footnote list or an article title with no sentence saying what the section was cited for, one was a table-of-authorities entry whose number also appears in the text, and one was plausible even to a reader ("peonage" cited in a jail-labour case).

Results: false alarms on correct documents

Why two quotation numbers. Published decisions sometimes misquote the cases they cite - an added or dropped word, no ellipsis - and old opinions carry scanning errors. A flag on such a quotation is correct, not a false alarm. We labelled every unconfirmed quotation in one run (blind28, 928 quotations) by hand: 27 were real differences in the source document, 31 came from the record or another source rather than the cited case, 20 were scanning errors (14 in our copy), and 18 were ours to fix. Counting only our own failures, 96.3% of correct quotations were confirmed; counting everything, 89.7%. We fixed the causes we found; the next hand-labelled run will be reported here.

Statutes: the AI meaning check

A section number changed to another real section (§ 2612 where § 2615 was meant) passes any "does it exist" check. LegiMatch's AI reads the sentence the statute is cited for next to the section's heading and text and flags a clear mismatch, suggesting the nearby section that fits. It also flags a U.S. Code section that has been repealed, and a section listed in the table of authorities that the brief never cites. In testing it also found statute mistakes courts and lawyers had made:

More: the statute section check.

Real mistakes it found in real filings

The same runs check the documents as filed, before anything is planted. Among the mistakes the brief check reported in published decisions and Supreme Court briefs:

Every test round

We ran 21 blind test rounds on 19 and 20 October 2026, each on documents nobody had used before: 1,635 documents, 43,046 real citations, 10,741 planted case-citation mistakes (plus 729 planted statute mistakes). We fixed what each round revealed and measured again on new documents, so the earlier rounds show where the checker started and the latest rounds show where it is now. Across all rounds, before and after the fixes, it caught 10,444 of 10,741 (97.2%) and called 11 of 43,046 real citations (0.03%) nonexistent; the two latest rounds, with every fix in place, caught 98.6% and called none nonexistent.

What we checked against

What these numbers don't show

• Coverage. Decisions held in our library (84% of the citations in the latest two runs: 4,347 of 5,185) get every check - name, year, court, pages, quotations, later treatment. Others are checked for existence, name and year against a citation index of 9,348,712 decisions decisions.
• Quotations given to the wrong case are the hardest case-citation mistake to catch (93%): the quoted words are real, just credited to another case.
• Statute swaps without a sentence - in a footnote list or a table of authorities - can't be judged by meaning.
• Meaning. The checker confirms a citation is real and correctly cited and that a quotation matches; whether the case supports the argument still needs a lawyer's reading.

How this compares

Westlaw (Quick Check) and Lexis (Brief Analysis) also check citations and quotations in a brief, but neither publishes an accuracy rate or a test method. We publish ours so you can judge it - and repeat it. See LegiMatch vs Westlaw vs Lexis.

Large research tools such as Lexis+ AI and Westlaw advertise accuracy without a published method; an independent Stanford study (2024) found their AI research answers contained errors in 17-33% of queries, which is a different task from citation checking. In our separate test of general chatbots on one 22-citation brief with eight planted errors, ChatGPT caught 5, Gemini 5, Copilot 4 and Perplexity 3; Claude (paid) and LegiMatch caught all 8 (full test).

Frequently asked questions

Can a citation checker replace reading the cases?

No. A checker tells you whether a citation is real and correctly cited, and whether a quotation matches the opinion. Whether the case supports your argument still needs a lawyer's reading.

Why are there two quotation numbers?

We count every quotation in real decisions, including ones the court itself misquoted and ones taken from the record. Excluding those, 96.3% were confirmed in our hand-labelled run; counting everything, about 90%. Part of the gap is real and we are closing it.

How often do you re-run this test?

Every week, on documents never used before, and after every change to the checker. We update the numbers here.