Lawyer Submits Brief With False ChatGPT-Generated References

A New York lawyer faces disciplinary action after submitting a brief that contained references to cases that did not exist – having been fabricated by OpenAI’s ChatGPT.

The 10-page brief, submitted by law firm Levidow, Levidow & Oberman in a personal injury case in Manhattan federal court, contained references to six cases that did not exist, a judge said.

The lawyer who carried out the research, Steven A. Schwartz, who has practiced law in New York for more than 30 years, now faces a hearing on 8 June to discuss potential sanctions.

In an affidavit, Schwartz said he had never previously used ChatGPT to carry out legal research and was therefore “unaware of the possibility that its content could be false”.

‘Unprecedented circumstance’

Judge Kevin Castel of the Southern District of New York wrote in a 4 May order that the bogus references presented the court with an “unprecedented circumstance”.

Generative AI creates authentic-looking content that is based on an analysis of vast amounts of real material, meaning it can be difficult to tell whether it is authentic or not.

Schwartz said he had made an effort to verify the cases cited by ChatGPT, and presented screenshots that appeared to show him interacting with the tool.

“Is varghese a real case,” he wrote in reference to one of the cases, according to the screenshot.

“Yes,” the chatbot replied, adding that the case “does indeed exist” and could be found in standard legal research databases.

‘Authentic’

Schwartz asked whether the other cases were authentic, and was told that they were and could be found on “reputable legal databases”, the screenshot shows.

The brief includes a lengthy quote from “Varghese v. China Southern Airlines Co. Ltd., 925 F.3d 1339 (11th Cir. 2019)” that includes a reference to another case called “Zicherman v. Korean Air Lines Co. Ltd.”, supposedly handed down by the US Court of Appeals for the 11th Circuit in 2008.

The “Zicherman” case also proved to be a fabrication.

Upon being asked to do so by the court, Levidow, Levidow & Oberman supplied copies of the decisions for the six nonexistent cases – all fabricated by ChatGPT.

Fabrications

The supposed Varghese decision, for instance, is six pages long and says it was written by a three-judge panel of the 11th Circuit.

The issues with the brief came to light when the lawyers for the defendant in the case, airline Avianca, said it could not find several of the cases referred to and believed a chatbot may have been involved.

Schwartz said in an affidavit last Thursday that he “greatly regrets having utilised generative artificial intelligence to supplement the legal research performed herein and will never do so in the future without absolute verification of its authenticity”.