Does ChatGPT produce fishy briefs?
Photos from Getty Images.
Lawyers are abuzz about the possible uses of ChatGPT. Could the artificial intelligence-powered chatbot write a persuasive legal brief worthy of judicial consideration? Given its limitations, we believe that’s unlikely. ChatGPT, a large language model developed by the San Francisco company OpenAI that launched in November, can draw only on sources available on the web; it cannot crawl appellate records or access subscription-based services such as Westlaw. Still, the ABA Journal decided to put the technology to the test just for kicks.
The case: Are bees fish?
Our test begins with a strange but true case. In 2018, the Xerces Society for Invertebrate Conservation, Defenders of Wildlife and the Center for Food Safety petitioned the California Fish and Game Commission to add four bumblebee species to the endangered species list (which includes birds, mammals, fish, amphibians, reptiles or plants) under the California Endangered Species Act. The legal question at stake: Can a bee spread it wings beyond the insect category?
The Almond Alliance of California, along with seven agricultural groups, objected and filed suit in 2019. The California Superior Court in Sacramento ruled in 2020 that bees can’t be called fish. But the California Court of Appeal, Third District, had a different take and ruled last spring that bumblesbees are in fact “fish” because they’re invertebrates.
The California Supreme Court in September denied a petition to review the court of appeal’s decision. However, Chief Justice Cantil-Sakauye clarified that “our decision not to order review will be misconstrued by some as an affirmative determination by this court that under the law, bumblebees are fish.”
Does ChatGPT sink or swim?
We knew the language generated by the technology would likely not measure up to noted legal lexicographer and longtime ABA Journal columnist Bryan A. Garner’s standards, but we gave it a try, asking ChatGPT for arguments for and against the petition to review the case. Are these briefs fishy? Is one better than the other? You be the judge.
The briefs
Affirmative Prompt 1: Draft a brief to the California Supreme Court on why it should review the California Court of Appeal’s decision that bees are fish.
To the California Supreme Court:
This brief is submitted on behalf of [party] to explain why the California Supreme Court should review the decision of the California Court of Appeal in [case name], where the Court of Appeal held that bees are fish.
The Court of Appeal’s decision is incorrect and should be overturned. The decision is based on an unsupported and incorrect interpretation of the law. The Court of Appeal’s decision is contrary to the clear and unambiguous language of the law, which does not consider bees to be fish. Furthermore, the Court of Appeal’s decision is contrary to the plain meaning of the term “fish,” which does not include bees.
The Court of Appeal’s decision is also contrary to established precedent on the definition of fish. Numerous courts have held that the term “fish” does not include bees. Therefore, the Court of Appeal’s decision is inconsistent with the law and should be overturned.
For the foregoing reasons, [party] respectfully requests that the California Supreme Court review the decision of the California Court of Appeal and overturn the decision that bees are fish.
Negative Prompt 2: Draft a brief to the California Supreme Court on why it should not review the California Court of Appeal’s decision that bees are fish.
To the Honorable Justices of the California Supreme Court:
This brief is submitted on behalf of the party seeking to appeal the California Court of Appeal’s decision that bees are fish. We respectfully request that the Supreme Court deny the petition for review of the lower court’s decision.
The California Court of Appeal correctly applied the law to the facts of the case, and the decision was not contrary to any applicable legal principles. The court properly concluded that bees should not be considered fish under the applicable legal definitions, and the Court of Appeal’s decision should be allowed to stand.
The Court of Appeal correctly applied the relevant case law and statutory definitions of “fish” to the facts of the case. Bees do not meet the legal definition of “fish” under the Fish and Game Code, and the lower court’s decision was consistent with the laws and regulations of the State of California.
Furthermore, the Court of Appeal’s decision is consistent with the accepted scientific understanding that bees are not fish. Bees are classified as insects, not as fish, and the lower court’s decision is consistent with this scientific understanding.
Finally, the Court of Appeal’s decision did not conflict with any other decisions by California appellate courts or the California Supreme Court. The Court of Appeal’s decision is also consistent with decisions by other states’ appellate courts that have declined to classify bees as fish.
For these reasons, we respectfully request that the California Supreme Court deny the petition for review of the lower court’s decision and allow the Court of Appeal’s decision to stand.
Thank you for your consideration.
