Anthopic’s law firm blames Claude hallucinations for errors • The Register

An legal professional defending AI agency Anthropic in a copyright case introduced by music publishers apologized to the courtroom on Thursday for quotation errors that slipped right into a submitting after utilizing the biz’s personal AI instrument, Claude, to format references.

The incident reinforces what’s changing into a sample in authorized tech: whereas AI fashions could be fine-tuned, folks maintain failing to confirm the chatbot’s output, regardless of the implications.

The flawed citations, or “hallucinations,” appeared in an April 30, 2025 declaration [PDF] from Anthropic knowledge scientist Olivia Chen in a copyright lawsuit music publishers filed in October 2023.

However Chen was not liable for introducing the errors, which appeared in footnotes 2 and three.

Ivana Dukanovic, an legal professional with Latham & Watkins, the agency defending Anthropic, said that after a colleague positioned a supporting supply for Chen’s testimony by way of Google search, she used Anthropic’s Claude mannequin to generate a formatted authorized quotation. Chen and protection legal professionals did not catch the errors in subsequent proofreading.

Sadly, though offering the right publication title, publication 12 months, and hyperlink to the offered supply, the returned quotation included an inaccurate title and incorrect authors

“After the Latham & Watkins group recognized the supply as potential further assist for Ms. Chen’s testimony, I requested Claude.ai to offer a correctly formatted authorized quotation for that supply utilizing the hyperlink to the right article,” defined Dukanovic in her Could 15, 2025 declaration [PDF].

“Sadly, though offering the right publication title, publication 12 months, and hyperlink to the offered supply, the returned quotation included an inaccurate title and incorrect authors.

“Our handbook quotation examine didn’t catch that error. Our quotation examine additionally missed further wording errors launched within the citations through the formatting course of utilizing Claude.ai.”

However Dukanovic pushed again in opposition to the suggestion from the plaintiff’s authorized group that Chen’s declaration was false.

“This was an embarrassing and unintentional mistake,” she stated in her submitting with the courtroom. “The article in query genuinely exists, was reviewed by Ms. Chen and helps her opinion on the correct margin of error to make use of for sampling. The insinuation that Ms. Chen’s opinion was influenced by false or fabricated info is thus incorrect. As is the insinuation that Ms. Chen lacks assist for her opinion.”

Dukanovic stated Latham & Watkins has applied procedures “to make sure that this doesn’t happen once more.”

The hallucinations of AI fashions maintain exhibiting up in courtroom filings.

Final week, in a plaintiff’s declare in opposition to insurance coverage agency State Farm (Jacquelyn Jackie Lacey v. State Farm Common Insurance coverage Firm et al), former Decide Michael R. Wilner, the Particular Grasp appointed to deal with the dispute, sanctioned [PDF] the plaintiff’s attorneys for deceptive him with AI-generated textual content. He directed the plaintiff’s authorized group to pay greater than $30,000 in courtroom prices that they would not have in any other case needed to bear.

After reviewing a supplemental temporary filed by the plaintiffs, Wilner discovered that “roughly 9 of the 27 authorized citations within the ten-page temporary have been incorrect not directly.”

Two of the citations, he stated, don’t exist, and a number of other cited phony judicial opinions.

Even with current advances, no fairly competent legal professional ought to out-source analysis and writing to [AI] – notably with none try and confirm the accuracy of that materials

“The legal professionals’ declarations in the end made clear that the supply of this drawback was the inappropriate use of, and reliance on, AI instruments,” Wilner wrote in his order.

Winer’s evaluation of the misstep is scathing. “I conclude that the legal professionals concerned in submitting the Authentic and Revised Briefs collectively acted in a way that was tantamount to unhealthy religion,” he wrote. “The preliminary, undisclosed use of AI merchandise to generate the primary draft of the temporary was flat-out improper.

“Even with current advances, no fairly competent legal professional ought to out-source analysis and writing to this know-how – notably with none try and confirm the accuracy of that materials. And sending that materials to different legal professionals with out disclosing its sketchy AI origins realistically put these professionals in hurt’s approach.”

In line with Wilner, courts are more and more referred to as upon to guage “the conduct of legal professionals and professional se litigants [representing themselves] who improperly use AI in submissions to judges.”

That is evident in instances like Mata v. Avianca, Inc, United States v. Hayes, and United States v. Cohen.

The choose tossed expert testimony [PDF] in one other case involving Minnesota Lawyer Common Keith Ellison – Kohls et al v. Ellison et al. – after studying that the professional’s submission to the courtroom contained AI falsehoods.

And when AI goes improper, it typically would not go properly for the legal professionals concerned. Attorneys from legislation agency Morgan & Morgan have been sanctioned [PDF] in February after a Wyoming federal choose discovered they submitted a submitting containing multiple fictitious case citations generated by the agency’s in-house AI instrument.

In his sanctions order, US District Decide Kelly Rankin made clear that attorneys are accountable in the event that they submit paperwork with AI-generated errors.

“An legal professional who indicators a doc certifies they made an inexpensive inquiry into the present legislation,” he wrote. “Whereas know-how continues to vary, this requirement stays the identical.”

One legislation prof believes that fines will not be sufficient – legal professionals who abuse AI ought to be disciplined personally.

“The quickest option to deter legal professionals from failing to quote examine their filings is for state bars to make the submission of hallucinated citations in courtroom pleadings, submitted with out cite checking by the legal professionals, grounds for disciplinary motion, together with potential suspension of bar licenses,” stated Edward Lee, a professor of legislation at Santa Clara College. “The courts’ financial sanctions alone won’t probably stem this follow.” ®