Why do lawyers keep using ChatGPT?

Each few weeks, it looks as if there’s a brand new headline a couple of lawyer getting in trouble for submitting filings containing, within the phrases of 1 choose, “bogus AI-generated analysis.” The main points differ, however the throughline is similar: an lawyer turns to a big language mannequin (LLM) like ChatGPT to assist them with authorized analysis (or worse, writing), the LLM hallucinates instances that don’t exist, and the lawyer is none the wiser till the choose or opposing counsel factors out their mistake. In some instances, together with an aviation lawsuit from 2023, attorneys have had to pay fines for submitting filings with AI-generated hallucinations. So why haven’t they stopped?

The reply principally comes right down to time crunches, and the best way AI has crept into almost each occupation. Authorized analysis databases like LexisNexis and Westlaw have AI integrations now. For attorneys juggling massive caseloads, AI can look like an extremely environment friendly assistant. Most attorneys aren’t essentially utilizing ChatGPT to put in writing their filings, however they’re more and more utilizing it and different LLMs for analysis. But many of those attorneys, like a lot of the general public, don’t perceive precisely what LLMs are or how they work. One lawyer who was sanctioned in 2023 stated he thought ChatGPT was a “tremendous search engine.” It took submitting a submitting with pretend citations to disclose that it’s extra like a random-phrase generator — one that might provide you with both right info or convincingly phrased nonsense.

Andrew Perlman, the dean of Suffolk College Regulation Faculty, argues many attorneys are utilizing AI instruments with out incident, and those who get caught with pretend citations are outliers. “I feel that what we’re seeing now — though these issues of hallucination are actual, and attorneys should take it very significantly and watch out about it — doesn’t imply that these instruments don’t have huge doable advantages and use instances for the supply of authorized companies,” Perlman stated. Authorized databases and analysis methods like Westlaw are incorporating AI companies.

In truth, 63 % of attorneys surveyed by Thomson Reuters in 2024 stated they’ve used AI prior to now, and 12 % stated they use it often. Respondents stated they use AI to put in writing summaries of case legislation and to analysis “case legislation, statutes, types or pattern language for orders.” The attorneys surveyed by Thomson Reuters see it as a time-saving software, and half of these surveyed stated “exploring the potential for implementing AI” at work is their highest precedence. “The position of a very good lawyer is as a ‘trusted advisor’ not as a producer of paperwork,” one respondent stated.

However as loads of current examples have proven, the paperwork produced by AI aren’t at all times correct, and in some instances aren’t actual in any respect.

In a single current high-profile case, attorneys for journalist Tim Burke, who was arrested for publishing unaired Fox Information footage in 2024, submitted a movement to dismiss the case towards him on First Modification grounds. After discovering that the submitting included “vital misrepresentations and misquotations of supposedly pertinent case legislation and historical past,” Decide Kathryn Kimball Mizelle, of Florida’s center district, ordered the movement to be stricken from the case file. Mizelle discovered 9 hallucinations within the doc, according to the Tampa Bay Times.

Mizelle in the end let Burke’s attorneys, Mark Rasch and Michael Maddux, submit a brand new movement. In a separate submitting explaining the errors, Rasch wrote that he “assumes sole and unique accountability for these errors.” Rasch stated he used the “deep research” feature on ChatGPT pro, which The Verge has beforehand examined with mixed results, in addition to Westlaw’s AI function.

Rasch isn’t alone. Legal professionals representing Anthropic lately admitted to using the company’s Claude AI to assist write an skilled witness declaration submitted as a part of the copyright infringement lawsuit introduced towards Anthropic by music publishers. That submitting included a quotation with an “inaccurate title and inaccurate authors.” Final December, misinformation skilled Jeff Hancock admitted he used ChatGPT to assist manage citations in a declaration he submitted in help of a Minnesota legislation regulating deepfake use. Hancock’s submitting included “two quotation errors, popularly known as ‘hallucinations,’” and incorrectly listed authors for an additional quotation.

These paperwork do, in reality, matter — a minimum of within the eyes of judges. In a current case, a California choose presiding over a case against State Farm was initially swayed by arguments in a quick, solely to search out that the case legislation cited was fully made up. “I learn their transient, was persuaded (or a minimum of intrigued) by the authorities that they cited, and regarded up the selections to be taught extra about them – solely to search out that they didn’t exist,” Decide Michael Wilner wrote.

Perlman stated there are a number of much less dangerous methods attorneys use generative AI of their work, together with discovering info in massive tranches of discovery paperwork, reviewing briefs or filings, and brainstorming doable arguments or doable opposing views. “I feel in nearly each job, there are methods through which generative AI may be helpful — not an alternative to attorneys’ judgment, not an alternative to the experience that attorneys deliver to the desk, however so as to complement what attorneys do and allow them to do their work higher, quicker, and cheaper,” Perlman stated.

However like anybody utilizing AI instruments, attorneys who depend on them to assist with authorized analysis and writing should be cautious to verify the work they produce, Perlman stated. A part of the issue is that attorneys usually discover themselves quick on time — a problem he says existed earlier than LLMs got here into the image. “Even earlier than the emergence of generative AI, attorneys would file paperwork with citations that didn’t actually handle the problem that they claimed to be addressing,” Perlman stated. “It was only a completely different sort of drawback. Typically when attorneys are rushed, they insert citations, they don’t correctly verify them; they don’t actually see if the case has been overturned or overruled.” (That stated, the instances do a minimum of sometimes exist.)

One other, extra insidious drawback is the truth that attorneys — like others who use LLMs to assist with analysis and writing — are too trusting of what AI produces. “I feel many individuals are lulled into a way of consolation with the output, as a result of it seems at first look to be so nicely crafted,” Perlman stated.

Alexander Kolodin, an election lawyer and Republican state consultant in Arizona, stated he treats ChatGPT as a junior-level affiliate. He’s additionally used ChatGPT to assist write laws. In 2024, he included AI textual content in a part of a invoice on deepfakes, having the LLM present the “baseline definition” of what deepfakes are after which “I, the human, added within the protections for human rights, issues like that it excludes comedy, satire, criticism, inventive expression, that sort of stuff,” Kolodin told The Guardian on the time. Kolodin stated he “could have” mentioned his use of ChatGPT with the invoice’s fundamental Democratic cosponsor however in any other case needed it to be “an Easter egg” within the invoice. The invoice handed into legislation.

Kolodin — who was sanctioned by the Arizona State Bar in 2020 for his involvement in lawsuits difficult the results of the 2020 election — has additionally used ChatGPT to put in writing first drafts of amendments, and informed The Verge he makes use of it for authorized analysis as nicely. To keep away from the hallucination drawback, he stated, he simply checks the citations to ensure they’re actual.

“You don’t simply sometimes ship out a junior affiliate’s work product with out checking the citations,” stated Kolodin. “It’s not simply machines that hallucinate; a junior affiliate might learn the case unsuitable, it doesn’t actually stand for the proposition cited anyway, no matter. You continue to should cite-check it, however it’s important to try this with an affiliate anyway, except they had been fairly skilled.”

Kolodin stated he makes use of each ChatGPT’s professional “deep analysis” software and the LexisNexis AI software. Like Westlaw, LexisNexis is a authorized analysis software primarily utilized by attorneys. Kolodin stated that in his expertise, it has a better hallucination price than ChatGPT, which he says has “gone down considerably over the previous yr.”

AI use amongst attorneys has change into so prevalent that in 2024, the American Bar Affiliation issued its first guidance on attorneys’ use of LLMs and different AI instruments.

Legal professionals who use AI instruments “have an obligation of competence, together with sustaining related technological competence, which requires an understanding of the evolving nature” of generative AI, the opinion reads. The steering advises attorneys to “purchase a basic understanding of the advantages and dangers of the GAI instruments” they use — or, in different phrases, to not assume that an LLM is a “tremendous search engine.” Attorneys also needs to weigh the confidentiality dangers of inputting info referring to their instances into LLMs and contemplate whether or not to inform their shoppers about their use of LLMs and different AI instruments, it states.

Perlman is bullish on attorneys’ use of AI. “I do assume that generative AI goes to be essentially the most impactful expertise the authorized occupation has ever seen and that attorneys can be anticipated to make use of these instruments sooner or later,” he stated. “I feel that in some unspecified time in the future, we’ll cease worrying concerning the competence of attorneys who use these instruments and begin worrying concerning the competence of attorneys who don’t.”

Others, together with one of many judges who sanctioned attorneys for submitting a submitting stuffed with AI-generated hallucinations, are extra skeptical. “Even with current advances,” Wilner wrote, “no moderately competent lawyer ought to out-source analysis and writing to this expertise — significantly with none try and confirm the accuracy of that materials.”