misterfergusson

Imagine a futuristic society where lawsuits are adjudicated within minutes. Accurately predicting the outcome of a legal action will change the way we adhere to rules and regulations. 

tl;dr
Lawyers are steeped in making predictions. A closely studied area of the law is known as Legal Judgment Prediction (LJP) and entails using computer models to aid in making legal-oriented predictions. These capabilities will be fueled and amplified via the advent of AI in the law.

Make sure to read the full paper titled Legal Judgment Predictions and AI by Dr. Lance B. Eliot at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3954615

We are in Mega-City One in the year 2099AD. The judiciary and law enforcement are one unit. Legal violations, disputes, infringements of social norms are enforced by street judges with a mandate to summarily arrest, convict, sentence, and execute criminals. Of course, this is the plot of Judge Joseph Dredd, but the technology in the year 2023AD is already on its way to making this dystopian vision a reality. 

Forecasting the legal outcome of a proceeding is a matter of data analytics, access to information, and the absence of process-disrupting events. In our current time, this is a job for counsel and legal professionals. As representatives of the courts, lawyers are experts in reading a situation and introducing some predictability to it by adopting a clear legal strategy. Ambiguity and human error, however, make this process hardly repeatable – let alone reliable for future legal action. 

Recent developments in the field of computer science, specifically around large-language models (LLM), natural language processing (NLP), retrieval augmented generation (RAG), and reinforced learning from human feedback (RLHF) have introduced technical capabilities to increase the quality of forecasting legal outcomes. It can be summarized as generative artificial intelligence (genAI). Crossfunctional efforts between computer science and legal academia coined this area of study “Legal Judgment Prediction” (LJP).  

The litigation analytics platform “Pre/Dicta” exemplifies the progress of LJP by achieving prediction accuracy in the 86% percentile. In other words, the platform can forecast the decision of a judge in nearly 9 out of 10 cases. As impressive as this result is, the author points out that sentient behavior is a far-fetched reality for current technologies, which are largely based on statistical models with access to vast amounts of data. The quality of the data, the methods leveraged to train the model, and the application determine the accuracy and quality of the prediction. Moreover, the author makes a case for incorporating forecasting milestones and focusing on those, rather than attempting to predict the final result of a judicial proceeding that is very much dependent on factors that are challenging to quantify in statistical models. For example, research from 2011 established the “Hungry Judge Effect” which in essence stated a judge’s ruling has a tendency to be conservative if it happens before the judge had a meal (or on an empty stomach near the end of a court session) versus the same case would see a more favorable verdict if the decision process took place after the judge’s hunger had been satisfied and his mental fatigue had been mitigated. 

Other factors that pose pitfalls for achieving near 100% prediction accuracy include the semantic alignment on “legal outcome”. In other words, what specifically is forecasted? The verdict of the district judge? The verdict of a district judge that will be challenged on appeal? Or perhaps the verdict and the sentencing procedure? Or something completely adjacent to the actual court proceedings? It might seem pedantic, but clarity around “what success looks like” is paramount when it comes to legal forecasting.  

While Mega-City One might still be a futuristic vision, our current technology is inching closer and closer to a “Minority Report” type of scenario where powerful, sentient or not, technologies churn through vast amounts of intelligence information and behavioral data to forecast and supplement human decision making. The real two questions for us as a human collective beyond borders will be: (1) how much control are we willing to delegate to machines? and (2) how do we rectify injustices once we lose control over the judiciary?