How AI legal research works, in plain English

Short answer: most AI legal research tools combine two parts: a search engine that finds relevant documents in a library of court decisions, and a language model that reads those documents and writes an answer or summary. How tightly the second part is tied to the first - called "grounding" - decides how reliable the tool is. A tool that can only return documents from its library cannot invent a case; a tool that lets the model write freely can.

Part 1: turning text into "meaning"

Computers compare numbers, not ideas. So AI search tools convert each passage of text into a long list of numbers called an embedding - a kind of fingerprint of what the passage means. Passages with similar meanings get similar fingerprints, even when they share no words.

That's why "the customer slipped on water in the store" can match "a shopper fell on a liquid near the entrance": the words differ, the fingerprints are close.

Part 2: retrieval - finding candidates

When you ask a question, the tool turns your question into a fingerprint too and finds the closest passages in its library. Good systems combine this semantic search with:

• keyword search, which still matters for exact terms, names and statutes
• filters, such as jurisdiction, court and date
• citation links, since courts cite related decisions

Merging several searches catches what any single one misses.

Part 3: re-ranking

The first pass returns dozens or hundreds of candidates. A second, slower step - often another AI model - reads the top candidates more carefully and re-orders them by how well they actually answer your question.

Part 4: generation - writing the answer

Finally, a language model writes a summary, an answer or an explanation, using the retrieved documents as its source material. This is called retrieval-augmented generation (RAG).

Here's the critical design question: is the model only allowed to cite what was retrieved?

A 2024 Stanford study of commercial legal AI tools found that even retrieval-based products produced incorrect or misgrounded answers on a meaningful share of questions - so "uses RAG" isn't a guarantee. What matters is whether you can see the source and check it.

What this means when you choose a tool

Ask the vendor:

1. Where do results come from - a database of decisions, or the model's own text?
2. Is every citation linked to the full opinion?
3. How current is the library, and which courts does it cover?
4. Can you see why a result ranked, or is it a black box?
5. Is my input used to train models?

Frequently asked questions

What is retrieval-augmented generation in legal research?

It's an approach where the AI first retrieves relevant documents - court decisions, statutes - from a library, then writes its answer using those documents as sources. It reduces, but doesn't eliminate, errors compared with a chatbot answering from memory.

Why do AI tools make up case citations?

Language models generate plausible text. Without a step that looks up and links real decisions, a model can produce a citation that has the right format but no real case behind it.

Is semantic search better than keyword search for case law?

They're good at different things. Semantic search finds cases with similar meaning in different words; keyword search finds exact terms and phrases. The strongest tools use both.

Can AI tell whether a case is still good law?

Some tools flag later negative treatment using citation data, but none is perfect. Always read the subsequent history yourself before relying on a decision.

Sources

• Lewis et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks", NeurIPS 2020
• Magesh, Surani, Dahl, Suzgun, Manning & Ho, "Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools", Stanford RegLab / HAI (2024)