Citing cases from an AI without checking they exist
The model will invent a case, a citation, and a quote, and present all three with the same calm confidence it uses for the real ones.
Not legal advice. Sally roasts behaviour and use-cases in general, never your specific situation, and nothing here replaces a real lawyer. The cases are real; what you do about them is between you and someone licensed to tell you.
Filing a brief with case citations a chatbot produced, without independently confirming each case is real.
Mata v. Avianca, Inc.
678 F. Supp. 3d 443 (S.D.N.Y. 2023) · US (S.D.N.Y.)
Two lawyers filed a brief citing six entirely fabricated ChatGPT-generated decisions, then doubled down when the court could not find them.
Judge Castel found subjective bad faith and imposed $5,000 in Rule 11 sanctions on the lawyers and their firm, and ordered them to notify the real judges falsely named as authors.
This is the founding cautionary tale of the entire genre. Lawyers filed a brief citing six entirely fabricated decisions a chatbot produced, complete with fake quotes and fake judges. Asking the chatbot "are these real?" and being told "yes" is not verification. It is the same machine lying to you twice.
An AI optimizes for text that looks like a citation, not for a citation that exists. A perfectly formatted reporter number with a plausible case name is the easiest thing in the world for it to generate and the hardest thing for a busy person to doubt.
“You asked the machine if its made-up cases were real, it said yes, and somehow that was the moment you stopped checking.”
- 01Pull every single case in a real legal database and read it before it goes anywhere near a filing.
- 02Use AI to suggest search terms and angles, not to supply authority you will sign your name under.
- 03If you cannot find the case in a reporter, it does not exist. Treat that as the default, not the exception.
Not legal advice. General commentary on a use-case, not your situation. Talk to a real lawyer before you act.