A very recent article by Adam Liptak in the New York Times draws attention to the actual practice in Wisconsin of using algorithm-driven predictions in sentencing, and to the pending attempt to otbain review of the practice by the US Supreme Court

In March, in a signal that the justices were intrigued by Mr. Loomis’s case, they asked the federal government to file a friend-of-the-court brief offering its views on whether the court should hear his appeal.

The report in Mr. Loomis’s case was produced by a product called Compas, sold by Northpointe Inc. It included a series of bar charts that assessed the risk that Mr. Loomis would commit more crimes.

The Compas report, a prosecutor told the trial judge, showed “a high risk of violence, high risk of recidivism, high pretrial risk.” The judge agreed, telling Mr. Loomis that “you’re identified, through the Compas assessment, as an individual who is a high risk to the community.”

The Wisconsin Supreme Court ruled against Mr. Loomis. The report added valuable information, it said, and Mr. Loomis would have gotten the same sentence based solely on the usual factors, including his crime — fleeing the police in a car — and his criminal history.

At the same time, the court seemed uneasy with using a secret algorithm to send a man to prison. Justice Ann Walsh Bradley, writing for the court, discussed, for instance, a report from ProPublica about Compas that concluded that black defendants in Broward County, Fla., “were far more likely than white defendants to be incorrectly judged to be at a higher rate of recidivism.”

There are so many issues bundled in here.

There is the issue of the use of algorithms at all in the making of predictions.  This is an issue of accuracy, fairness and legitimacy.

There is the issue of transparency.  The idea of not knowing the algorithm’s factors and logic seems bizarre, particularly when defended in commercial terms.  There is the issue of powerlessness of defendants and others somehow having no control of the fact process.

Finally, there is the deeply disturbing issue of embedded bias, which may be impossible to correct for.  I will deal with the embedded bias issue in more detail in a future post.

Firstly, as to the use of algorithms in making predictions there is significant evidence that they increase accuracy and fairness.  To be specific, studies have shown that algorithm productions and decisions can be more reliable and less prone to bias than human predictions and decisions.

In this research, statistical methods applied to Terry stops showed that cops using a very simple algorithm tool would make far fewer nonproductive stops than those relying on their fast intuition. To be specific:

Secondly, as to transparency, let me describe what we did on this front at the Midtown Community Court. When a judge asked us if we could develop an algorithm predicting compliance with alternative sanctions, some of us demurred, not because of its technical difficulty, but because of the fear of people, in effect, being sentenced based on the non-compliance of others.  Then the judge said something that will echo with me for the rest of my life: “I just do not want to set people up for failure.”

Ultimately we built a system with three major features: 1. the probabilities were based on actual data and factors shown by regression analysis to be critical; 2. the factors impact were shown in histograms so that these factors could become part of the conversation.  Counsel might for example, point out that while a defendant did not have a formal address, he did have a place to live.  In such a case, counsel would ask the judge to change the homelessness setting.  Then, you could literally watch the histograms bounce around to show the new compliance projections.  Finally, we gave the judges compliance support tools, enabling them, for example, to order reminder phone calls to the defendant.

The conclusion I draw from this is that transparency, and indeed then then enabled  discussion, is critical to the effectiveness and legitimacy of these tools.  Proprietary commercial interests can be no excuse for secret government.  Moreover, confidentiality of algorithms is not necessarily required for protection of intellectual property, the law can protect such interests without secrecy.  Most patents are public.

Finally, as to embedded bias, let me now in this post just note how many deeply entangled levels such inevitably has, preserving and projecting into the future, the harms of the past.  The question is whether such embeds are better or worse that the bias of individualized discretion.

In a future posting, I will attempt to lay out some principles that should be followed in developing and using such predictive algorithms in the justice system.