Hallucination Has Structure: The Lawyer Who Cited Fake Cases
This is Part 5 of our 16-week series on Context Degradation—the hidden failure modes that break AI systems before anyone notices. The Case of the Nonexistent Cases
In May 2023, attorney Steven Schwartz filed a brief in federal court containing citations to six cases supporting his client's argument.
Varghese v. China Southern Airlines Shaboon v. Egyptair Petersen v. Iran Air Martinez v. Delta Airlines Estate of Durden v. KLM Royal Dutch Airlines Miller v. United Airlines
The judge couldn't find any of them.
Because none of them existed.
Schwartz had used ChatGPT to research case law. ChatGPT had generated plausible-sounding but entirely fictitious cases, complete with citations, court names, and legal reasoning.
When confronted, Schwartz asked ChatGPT if the cases were real. ChatGPT confidently confirmed they were.
The judge sanctioned Schwartz and his firm. The legal profession panicked. AI critics declared vindication.
But the most important lesson got lost in the headlines: the hallucinations weren't random.
The Structure of Fake
Here's what ChatGPT generated for one fake case:
Varghese v. China Southern Airlines, 925 F.3d 1339 (11th Cir. 2019)
That's not random characters. It's a perfectly formatted federal case citation:
Party name v. Party name Volume number Reporter abbreviation Page number Court abbreviation Year
The fake case followed real case naming conventions. It had plausible party names for an aviation dispute. It cited a real federal reporter. It used a real circuit court. It gave a reasonable year.
The hallucination was structurally correct and semantically plausible. That's exactly why it was dangerous—and exactly why it's detectable.
High Confidence + Low Reliability = Hallucination
In our degradation taxonomy, HALLUCINATION is specifically:
High α (alpha): The model is confident in its output Low ω (omega): The output doesn't reliably correspond to verifiable reality
This combination is the signature of hallucination. The model isn't uncertain and guessing—it's certain and wrong.
Why does this happen? Because language models optimize for plausibility, not factuality. They learn what sounds right, not what is right.
A case citation that follows the correct format sounds right. Whether the case exists is a different question—one the model has no mechanism to verify.
Why "Just Add Retrieval" Doesn't Fully Solve This
The obvious fix for hallucination is RAG: ground the model in real documents, and it won't make things up.
This helps. But it doesn't fully solve the problem for several reasons:
1. The model can still hallucinate beyond the documents RAG provides context. It doesn't prevent the model from extrapolating, interpolating, or fabricating details not in that context.
2. Retrieval can fail If the relevant document isn't retrieved, the model falls back to parametric knowledge—which can hallucinate.
3. The model can misread its context "Lost in the Middle" (Week 2) showed that models don't reliably use all their context. They can hallucinate even with the right answer present.
4. Confidence doesn't decrease appropriately RAG-augmented models are often just as confident in wrong answers as right ones. The retrieval feels like grounding even when it isn't.
The Lawyer's Tragic Error
Schwartz made a comprehensible mistake. He asked ChatGPT for cases. ChatGPT gave him cases that looked real. He asked ChatGPT if they were real. ChatGPT said yes.
This is the HALLUCINATION failure mode in action:
High confidence: ChatGPT expressed certainty at every step Low reliability: The cases didn't exist No signal: Nothing in the interaction indicated the gap
Schwartz trusted the confidence. He had no way to detect the low reliability short of manually checking each citation (which, admittedly, is basic legal research practice).
Detecting Hallucination Before It Ships
The Schwartz case illustrates why output-based detection is too late. By the time someone checks whether the cases are real, the brief is already filed.
What we need is pre-generation detection. Before the model outputs a confident answer, we need to know:
Does the context support this level of confidence? Are there verification signals in the retrieved content? Is this the kind of claim where hallucination risk is elevated?
A Context Quality Certificate measures the gap between alpha (confidence) and omega (reliability):
High α, High ω: Confident and reliable → Proceed Low α, Low ω: Uncertain and unreliable → Retrieve more or decline Low α, High ω: Uncertain but reliable → Boost confidence, proceed High α, Low ω: Confident but unreliable → HALLUCINATION RISK → Require verification
That fourth quadrant is where hallucination lives. Detecting it before generation enables intervention.
Why Hallucination Has Structure
The reason hallucination is detectable is that it follows patterns:
Structural plausibility: Hallucinated content follows format conventions (like case citations)
Semantic plausibility: Hallucinated content…