ChatGPT-using lawyer punished for citing fake testimony from made-up witnesses

Many lawyers have been caught citing fake cases in briefs or inaccurately describing real cases. While Aarons didn’t cite fake cases, he inaccurately described real ones and cited fake testimony. Aarons told the state Supreme Court at a hearing on August 21 that he fed a computer-generated transcript of the murder trial and other documents related to the case into ChatGPT, which outputted the fake quotes. “It’s of little comfort to know that my stupidity is what brings us together this afternoon,” Aarons told the court. Aarons admitted his brief quoted “several witnesses who were never called at trial,” another “witness who was called but the brief got the name wrong,” and that his brief inaccurately described precedents. Aarons indicated that he used a version of ChatGPT powered by the OpenAI o3 model, which was released earlier in 2025. “I assumed that it generated a bulletproof summary of proceedings,” he said at the hearing, explaining that he thought it would be accurate because of how widespread AI use is in the legal and medical fields. Aarons was barred from appearing before the New Mexico Supreme Court pending the outcome of any disciplinary board investigation and proceedings. He was fined $5,000, to be paid to the State Bar of New Mexico Client Protection Fund, but more penalties could come out of the disciplinary proceedings ordered by the court. The state Supreme Court ordered the public defender office to appoint a new lawyer for the defendant, struck all previous briefs from the record, and said the case will proceed in the court’s 2026-27 term. Justices lambasted Aarons during last month’s hearing. They expressed surprise that Aarons didn’t know AI tools could generate false information and pointed out that attorneys must verify the accuracy of information no matter where it comes from. Whether a lawyer gets help from an AI tool, a law student, or a fellow attorney, the lawyer signing the brief must attest to its accuracy, they said. Justice C. Shannon Bacon was particularly withering in her criticism of Aarons. She told Aarons that there are “at least eight or nine provisions in the code of conduct that you violated.” Bacon said she was “really struggling with” Aarons’ claim to be unaware of AI hallucinations, noting that her 13-year-old nephew and 75-year-old stepmother are both aware of the problem. “So counsel, do you watch the news? Do you listen to the radio? Do you read anything about what’s going on in the world? Because the problem with lawyers relying on AI hallucinations is an above-the-fold story every single day,” she said. “So either you buried your head in the sand—and that’s a choice to do that, an intentional choice to be uninformed—or you took a gamble, and neither of those are consistent with the code of conduct.” Aarons responded that he submitted the brief a year ago and “a lot has come out in the last year.” However, the problem of lawyers using AI in briefs that cite made-up cases has been in the news regularly for well over three years. Aarons added that the cases he cited in his brief were not fake, although his brief described them inaccurately. Bacon responded that there’s no “material distinction” between inaccurately describing a real case and citing a fake one, as the rules about candor to the court apply “with equal force” either way. “I believe I did confirm” that the trial transcript was accurate, he said when asked if he confirmed its accuracy before inputting it into ChatGPT. “The problem wasn’t in the transcript. It was when I loaded it and all the other information” into ChatGPT, he said. In addition to the transcript, Aarons said he put “the record proper, the statement of issues, and some of the discovery” into ChatGPT. Aarons said he hoped the court would take steps to prevent other lawyers from making the same mistake he did, such as by issuing a standing order requiring that briefs contain a certificate of compliance regarding the use of AI. Justices treated Aaron’s suggestion as a distraction from the main problem, his failure to verify the accuracy of the brief. “Assume with me that you had relied on the work of a first-year lawyer that was working for you and that they had just made stuff up… and you signed it. You’d be in the same exact soup you are right now,” Bacon said. “So the idea that, the suggestion in your briefing that because the court didn’t tell you at the time that you did this, ‘be careful,’ it somehow relieves you of obligation, falls on absolute deaf ears because the rules of professional conduct already tell you what your obligation is.” “The other thing that’s completely missing from your response is anything about what this has done to your client,” Bacon went on to say. “That’s who I’m worried about. You have presented briefing to the court that we cannot rely on, and your client is the one suffering because of this far more than you will ever suffer.” Chief Justice Julie Vargas similarly told Aarons, “I’m really interested with the approach you’re taking in this hearing. You seem to be telling us all the policy that we’ve been thinking about for years about what to do with AI, but you’re not talking about how to address the situation that’s in front of us, which has significant impacts on a criminal defendant who is in custody, who’s going to stay in custody until we resolve this matter. And I really don’t care about the policy concerns right now. I care about what we’re going to do with your client and what we should do in this circumstance based on the behavior that you showed us.” Justice Michael Vigil said there’s nothing wrong with using AI to help write a legal argument as long as the lawyer verifies its accuracy. “It doesn’t matter what the tool is. It doesn’t matter how advanced the AI-generated program is or what improvements they make, whatever,” he said. “It doesn’t matter whether you use a C-student lawyer or a A-student lawyer [if] you didn’t check their work before you filed the brief. That’s the issue. And I wish, or I’m hoping you would focus on that, take responsibility for it.” Aarons replied, “I hope I don’t present myself as not accepting responsibility. I mean, my only hope is that the mistakes that I made that somehow can prevent someone else down the line from repeating my mistake because the AI capacity is amazing.” Bacon criticized Aarons for not disclosing the problem to his client. “I have not. He’s a Spanish speaker and I wanted to see what happened today,” Aarons told Bacon when she asked if he told Sandoval about the problem. Aarons said he was hired by Sandoval’s brother and cousin, and that he told them there was a problem with a brief. “They’re aware that there was a problem with the first brief that I filed and that the court’s looking at that. That’s pretty much it,” he said. Bacon replied, “You didn’t say, ‘I relied on ChatGPT and it submitted lies to the court and the court is mad at me and now there’s a hearing.’ I’m guessing you didn’t tell your client’s family members that.” “I didn’t talk about ChatGPT,” Aarons said. “I just said there was a problem with the brief that I filed.” Bacon told Aarons that his client needed to know the extent of the problem so he can make an informed decision “about whether you’re the right person to represent him.” Bacon asked Aarons whether he was working on the case on a public defender contract or as private counsel. Aarons answered that he was hired as private counsel but hadn’t been paid yet. “I’m not sure that’s going to happen,” Bacon replied.