AI program earned passing bar exam scores on evidence and torts; can it work in court?
Image from Shutterstock.
Artificial intelligence programs are making inroads in the legal field, showing proficiency in two sections of a practice bar exam and getting used for the first time in traffic court.
Reuters and Law.com have the bar exam stories. An artificial intelligence program called ChatGPT-3.5 managed to pass evidence and torts sections of a multiple-choice, multistate bar exam.
According to Reuters, ChatGPT “generates sophisticated, human-like responses based on requests from users and mountains of data, including from legal texts.” An ABAJournal.com story said ChatGPT-3.5 “has the uncanny ability to generate incredibly detailed responses that even incorporate the requested tone or perspective, whether it’s drafting social media posts, a college essay or an oddly specific folk song.”
ChatGPT didn’t pass the overall bar exam, according to a Dec. 31 paper describing the OpenAI chatbot’s performance by Daniel Martin Katz, a professor at the Chicago Kent College of Law, and Michael Bommarito, a professor at the Michigan State University College of Law. Overall, it was correct 50.3% of the time. Real test-takers get 68% of the questions right.
The bar exam results are leading to fears by some law professors that students will use the program to cheat on take-home tests. But others, such as Andrew Perlman, dean of the Suffolk University Law School, said tools such as ChatGPT can be put to good use, according to Reuters.
Perlman said research and writing classes for 1Ls could cover AI tools, and in the future, lawyers could rely on chatbots for initial drafts of documents. When Perlman tried out ChatGPT, he asked it to complete a series of tasks, including drafting deposition questions and explaining why the U.S. Supreme Court decision on same-sex marriage should not be overturned, according to a prior story by Reuters.
Perlman said the bot “isn’t ready for prime time.” But according to Reuters, there were the makings of an advocate in its response to the same-sex marriage question. The bot talked about equal protection of the law and the fundamental right to marry, citing Loving v. Virginia.
The founder of DoNotPay, a startup founded to help motorists contest their traffic tickets, said he is planning to use an open-source AI program in traffic court.
Joshua Browder, a 2017 ABA Journal Legal Rebel, said he will use GPT-J to assist two defendants next month when they appear before judges to fight speeding tickets, report USA Today, New Scientist and Politico. DoNotPay will cover fines and will compensate the defendants, according to USA Today.
One of the defendants will appear in a Zoom hearing, Browder told Politico. He is even considering letting the AI program do the talking, although he conceded “that could get us into a lot of trouble.” The defendant in the courtroom will wear earbuds to hear the chatbot’s legal arguments.
Browder has also addressed concerns that use of the AI program amounts to the unauthorized practice of law. The traffic court hearings are taking place in jurisdictions where his plan is “not outright illegal,” Browder told Politico.
Browder is so optimistic that he tweeted an offer of $1 million to anyone who will wear earbuds and repeat arguments by his “robot lawyer” before the Supreme Court, according to USA Today and Gizmodo via How Appealing.
Browder later told Gizmodo that the $1 million would be a donation to the lawyer, and his plan would only be enacted if the Supreme Court allowed it. He said he hadn’t heard from any Supreme Court litigators, but he has received “very serious offers” from other appellate lawyers.
