New game lets players train AI to spot legal issues
Image from Shutterstock.
Got a free minute? There’s a new game that will help train an artificial intelligence model to spot legal issues and help close the access-to-justice gap.
Called Learned Hands—yes, it’s a pun—the game takes 75,000 legal questions posted on Reddit dealing with family, consumer, criminal and other legal issues and asks the user to determine what the issue is.
While conjuring up nightmares of the first-year in law school for many lawyers, David Colarusso says it’s for a good cause.
“It’s an opportunity for attorneys to take their downtime to train machine learning algorithms to help access-to-justice issues,” says Colarusso, director of Suffolk University Law School’s Legal Innovation and Technology (LIT) Lab and partner on this project with the Stanford Legal Design Lab.
As the ABA Journal reported in May, the goal of the project is to create a taxonomy of legal questions as asked by laypeople that can be used to label datasets that machine-learning models can be trained on.
Colarusso explained that this project is necessary because there is a “matchmaking problem” when it comes to websites providing legal information. The current dominant model is listing topics based on legal terms of art like “family law.”
By taking tens of thousands of questions covering several dozen legal issues, Colarusso said the project aims to build a training dataset that can help create “algorithmically driven issue spotting” to assist online legal help portals more accurately connect information and resources to users and diminish the access-to-justice gap.
In one example from the game, a person posted about a speeding ticket they had received in Alabama. Discussing the incident, the author mentions that his wife and kids were in the car. A machine trained to correlate “wife” and “kids” with family law would miscategorized this question. An attorney, or most literate people, would be able to discern that this was not a family law issue.
At the bottom of the screen the user was asked, “Do you see a legal issue around Family in this post?” followed by two boxes, one approving and the other disapproving. As answers pile up saying this is not a family law issue the question can then be labeled and used to train a machine to spot a traffic law issue.
With more people playing the game, Colarusso explains, the tool uses the wisdom of the crowd to determine the correct labels for each question. This means that each correct label will represent, at a minimum, a 95 percent probability that the label is rightly applied.
To motivate players, each answer provided in the game gives the user points. Those points are compared to the correct answers, which gives the user a total score. The game hopes that people will compete against each other to be a top issue spotter—bragging rights are the prize.
Once sets are properly labeled, the team behind the project will release the data publicly so that other researchers can use it to train other machine learning models. The project is funded by the PEW Charitable Trusts.
The success of the game will impact the success of the overall project. However, if it proves popular, it could be “a goldmine for a long time to come,” says Colarusso.
