Summarizing their work, Daniel Katz, Michael Bommarito and Josh Blackman wrote in A General Approach for Predicting the Behavior of the Supreme Court of the United States:
[W]e offer the first generalized, consistent and out-of-sample applicable machine learning model for predicting decisions of the Supreme Court of the United States. Casting predictions over nearly two centuries, our model achieves 70.2% accuracy at the case outcome level and 71.9% at the justice vote level. More recently over the past century, we outperform an in-sample optimized null model by nearly 5 %. Among other things, we believe such improvements in modeling should be of interest to court observers, litigants, citizens and markets. Indeed, with respect to markets, given judicial decisions can impact publicly traded companies, as highlighted in [Katz DM, Bommarito MJ, Soellinger T, Chen JM. Law on the Market? Evaluating the Securities Market Impact of Supreme Court Decisions 2015], even modest gains in prediction can produce significant financial rewards.
Here’s the abstract:
Building on developments in machine learning and prior work in the science of judicial prediction, we construct a model designed to predict the behavior of the Supreme Court of the United States in a generalized, out-of-sample context. To do so, we develop a time evolving random forest classifier which leverages some unique feature engineering to predict more than 240,000 justice votes and 28,000 cases outcomes over nearly two centuries (1816-2015). Using only data available prior to decision, our model outperforms null (baseline) models at both the justice and case level under both parametric and non-parametric tests. Over nearly two centuries, we achieve 70.2% accuracy at the case outcome level and 71.9% at the justice vote level. More recently, over the past century, we outperform an in-sample optimized null model by nearly 5%. Our performance is consistent with, and improves on the general level of prediction demonstrated by prior work; however, our model is distinctive because it can be applied out-of-sample to the entire past and future of the Court, not a single term. Our results represent an important advance for the science of quantitative legal prediction and portend a range of other potential applications.