Hallucination

What a citation gate is, and why “grounded” AI still fabricates

A citation gate is a check that runs after a draft exists and before it leaves the building. It asks two questions about every authority in that draft, and it has to ask both.

The first is an existence check. Is there a case at this reporter and page, a statute at this section, an exhibit at this Bates number? That is a lookup. The string either resolves to a document or it does not.

The second is a support check. Does that document hold what the sentence attached to it says it holds? That is not a lookup. It requires reading the cited passage and deciding whether it carries the proposition the writer hung on it.

Most products that advertise verified citations run the first check. Nearly all of the interesting failures are in the second.

The field’s own word for it is “misgrounded”

Researchers at Stanford’s RegLab and the Institute for Human-Centered AI ran the first preregistered evaluation of the AI legal research tools sold by LexisNexis and Thomson Reuters [1]. They sorted the failures into two kinds. A response is incorrect when it states the law wrongly. A response is misgrounded when it states the law correctly but cites a source that does not in fact support the claim [2].

A citation might be “hallucination-free” in the narrowest sense that the citation exists, but that is not the only thing that matters. [2]

They argued the misgrounded answer may be the more dangerous of the two, because it survives the check a busy reader actually performs: click the cite, see a real case, move on [2].

The rates were not marginal. In the study’s own word, Lexis+ AI and Ask Practical Law AI hallucinated more than 17 percent of the time, and Westlaw’s AI-Assisted Research more than 34 percent of the time — hallucinated in its sense, which covers both kinds: incorrect and misgrounded [2]. These are retrieval-based products, sold to lawyers, marketed in the vendors’ own words as “avoid[ing]” hallucinations and guaranteeing “hallucination-free” legal citations [1].

What the second failure looks like in an order

In Coomer v. Lindell, Judge Nina Y. Wang catalogued what was wrong with a brief: misquotes of cited cases; misrepresentations of principles of law associated with cited cases, including discussions of legal principles that simply do not appear within such decisions; misstatements about the law of the district; and, the order says, most egregiously, citation of cases that do not exist [3]. Read the list in order. Invention is the last item, not the first.

Lacey v. State Farm is the cleaner illustration. A special master reviewing a ten-page brief found that roughly nine of its 27 citations were wrong in some way. At least two of the authorities did not exist at all, and several quotations attributed to real opinions were phony [4]. So an existence check clears most of that brief. Seven of the nine bad cites point at cases you can pull off the shelf.

The special master described what the defect cost him:

I read their brief, was persuaded (or at least intrigued) by the authorities that they cited, and looked up the decisions to learn more about them — only to find that they didn’t exist. [4]

He was doing the existence check by hand, which is the check a machine can already do. The quotations that misdescribed real cases are the ones that would have gone through.

Why retrieval does not close the gap

Retrieval changes what is in front of the model. It does not change what the model does next. The system finds documents, places them in the prompt, and then writes a new sentence. That sentence is generated, so it can still be invented, and the citation appended to it is a claim about the retrieved document rather than a measurement of it. Nothing in the pipeline has compared the two.

What a gate has to do to earn the name

Four steps, in order, for every proposition in the draft:

  1. Resolve. Turn the citation into one specific document, not a title that looks right.
  2. Locate. Find the passage relied on — the page, the paragraph, the holding — not the case in general.
  3. Test. Decide whether that passage supports this proposition, in this jurisdiction, as of this date.
  4. Fail closed. If it cannot resolve, cannot locate, or cannot support, the sentence does not ship. The gap is named where the sentence would have been.

Step four is the one that costs something, and it is the one that makes the other three worth running. A checker that flags a problem and prints the sentence anyway has only moved the work back to you.

Why the difference is yours to care about

ABA Formal Opinion 512 puts it in the language of Rule 1.1: using a generative AI tool without an appropriate degree of independent verification or review of its output could violate the duty of competence [5]. A tool that verifies only existence returns the harder half of the job while leaving the impression the job is done. That impression is the risk.

This is the distinction Apodicta was built around: the gate is code outside the model, it runs on the support question as well as the existence question, and it renders nothing it can’t ground. What fails a gate is held and named, on the page, rather than quietly printed.

Sources

  1. Hallucination-Free? Assessing the Reliability of Leading AI Legal Research ToolsMagesh, Surani, Dahl, Suzgun, Manning & Ho (Stanford RegLab / HAI), arXiv:2405.20362 · 30 May 2024
  2. AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking QueriesStanford Institute for Human-Centered AI · 23 May 2024, updated 30 May 2024
  3. Order on Order to Show Cause, Coomer v. Lindell, No. 1:22-cv-01129 (D. Colo.)U.S. District Court, District of Colorado (via CourtListener) · 7 July 2025
  4. Order re Sanctions, Lacey v. State Farm General Ins. Co., No. 2:24-cv-05205 (C.D. Cal.)U.S. District Court, Central District of California (via CourtListener) · 6 May 2025
  5. Formal Opinion 512: Generative Artificial Intelligence ToolsABA Standing Committee on Ethics and Professional Responsibility · 29 July 2024

More: all articles · questions defenders ask · the numbers, with their receipts

See the gate on your own record.

Show me on my files