How we test LegiMatch's search accuracy (and our current results)
By LegiMatch Research Team · Published October 14, 2026 · Updated October 21, 2026 · 3 min read
Short answer: we test search quality four ways - realistic lawyer questions, rewritten versions of real decisions, the citations judges actually relied on, and finding known cases and quotations. In our latest test (20 October 2026), when we described 80 real decisions' facts in new words, LegiMatch returned the original decision first in 51 of 80 and in the top 10 in 68 of 80 (85%). Pasting a real quotation found the case it came from in 39 of 40 tries, in under a second. We re-run these tests as the library grows and publish the new numbers.
Why publish this
AI legal research tools rarely publish how often they find the right case. A 2024 Stanford study of commercial legal AI tools found that even leading products produced incorrect or unsupported answers on a substantial share of test questions - which is why we think lawyers deserve numbers, including the unflattering ones.
Test 1: realistic lawyer questions
Fourteen fact patterns written the way an attorney would describe a matter (a slip-and-fall in a grocery store, a DWI suppression issue, a whistleblower retaliation claim, and so on). We check the top five results.
| Measure | Result |
|---|---|
| Top-five results in the right area of law | about 91% |
| Top five include a decision on the expected legal issue | 14 of 14 |
Test 2: rewritten real decisions
We take a real decision, have its facts rewritten as a lawyer would describe a new client's matter - new names, new places, different wording - and search for it the way the site does, with AI reading the matter and re-ranking the candidates. A good search should find the original. The October 2026 run used 80 decisions.
| Original found at #1 | In top 3 | In top 10 |
|---|---|---|
| 51 of 80 | 62 of 80 | 68 of 80 (85%) |
Details: our similar-case test.
Test 3: the citations judges relied on
For opinions that cite other decisions in our library, we search with the citing opinion's facts and check whether the cases the judge actually cited appear in our top 10. This is a deliberately hard test: judges often cite cases for side points, such as the standard of review, that aren't factually similar at all.
| Measure | Result |
|---|---|
| Cited decisions found in our top 10 | 33% (up from 26% before our October 2026 improvements) |
Test 4: finding a known case and a quotation's source
| Search | Result |
|---|---|
| By citation (e.g. "558 U.S. 310") | 58 of 60 (96.7%) in our latest audit |
| By case name | 58 of 60 (96.7%) |
| A real quotation lawyers and judges took from earlier cases | Source case found in 39 of 40 |
| A random sentence from an opinion | Found in 29 of 29 |
What these numbers don't tell you
- The library keeps growing. A larger library adds more relevant decisions and more competition, so we re-test and publish updated numbers.
- "Found the original" is a proxy. In real research you want useful precedent, which may not be one specific case.
- No test replaces reading. Every result must be read and verified before you rely on it.
What's next
We ask attorneys who use LegiMatch to rate results as useful or not useful. That rating - not our internal tests - is the measure we care about most, and we'll publish it.
Frequently asked questions
How accurate is AI legal research?
It varies widely by tool and task. Independent research has found meaningful error rates even in leading commercial tools, so results must always be verified. Tools that return only real, linked decisions avoid invented citations but can still miss relevant cases.
Does LegiMatch ever return cases that don't exist?
No. Every result is a decision stored in our database, linked to the court's text. The AI can describe a case incorrectly, which is why explanations are labeled and the full opinion is one click away.
How often do you re-test?
On every change to the ranking, and with fresh published numbers each quarter.
Sources
- Magesh, Surani, Dahl, Suzgun, Manning & Ho, "Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools", Stanford RegLab / HAI (2024)
This article is general information about legal research, not legal advice. Verify every authority before relying on it.
All articlesMore from the blog
- Can AI find a case from its facts alone? A published test of similar-case searchWe rewrote 80 real court decisions in plain words, removed every name and citation, and measured how often LegiMatch Case Match finds the exact original among 4.7 million U.S. court decisions - with the method, so anyone can repeat it.
- Cases you may have missed - finding authority your brief doesn't citeAfter LegiMatch checks your brief, one click runs a fact-based search on the brief itself and lists strong matching decisions you haven't cited - how it works, what it costs and how to use the results.
- How accurate is LegiMatch Brief Check? A blind test on real court documentsOur blind test of the brief checker, with a detection rate for each kind of citation mistake - wrong court, year, pin cite, case name, quotation, invented case, wrong volume, swapped citations and statutes - plus how often it wrongly flags correct citations, measured on real court decisions and U.S. Supreme Court briefs nobody had used before.