As part of my series on algorithm bias, I want to offer what I think is a potentially revolutionary take on using algorithms not just to minimize, but actually to compensate for, bias and discrimination.

The core idea is that if algorithms, as it now becoming to be understood, can reflect the bias on a society, then they can also be used to measure the extent of the impact in a particular situation and then build in the appropriate compensation into the algorithm.  In contrast to the traditional and constitutionally disfavored “affirmative action,” this is not general race or gender-based “bias,” but rather the application of neutral rules in a highly individualized context.

At the risk of being highly controversial, even a supporter of “affirmative action” could not comfortably assert that there have never ever ever been instances of unfair advantage. (Although it is very safe to say that those seeing such advantage often fail completely to understand the extent to which bias, particularly over time, has disadvantaged in the individual case.)  Conversely, it would be a very arrogant opponent of affirmative action who would assert that such an approach was in fact unfair in every case.  The overall problem is that confirmation bias means that it is really easy to become convinced of the unfairness of affirmative action, and then take that out on those who are not even its actual beneficiaries, in any sense.

In any event, the broader point is that when algorithms pick up and reflect the history of bias and its impact and make predictions based on those patters, they are, of course perpetuating bias and essentially making it impossible overcome.  Recent media coverage has focused on one small example, in which AI analysis of words has shown that works associated with some races occur more frequently near to “positive” words, and vice versa.

More practically, however, I am worried by big data developing things like career success predictions, or trial and sentencing outcomes, based on apparently neutral factors like residence, or job history, when those predictions are themselves heavily based on histories of bias (both affirmative and negative in both senses — who at Harvard is not the beneficiary of some form of affirmative action?)

Most algorithms development systems are, hopefully, savvy enough to intentionally forbid the use of illegal factors in the algorithm.  But, if you allow those factors to be included in the development of an alternative algorithm, you are half way to a measure of the extent to which prior bias has contributed a person’s current future opportunities.

Let me give an example. Lets say that an algorithm ignores forbidden factors and gives a 50% chance of completing probation.  If we then run a forbidden factor sensitive algorithm (i.e. we now do include race, gender, etc, for research purposes only) and it gives us only the same 50% chance of completing probation, then we actually know that all of the persons risk comes from long term forbidden factor related issues rather than the decision-maker’s knowledge of person’s race.  I would think such a result would be a powerful wake up call to anyone.  Moreover, the difference in the contributions of components to the score would give us data on what was making the biggest difference — housing schools, policing, etc.

If the numbers are not the same, then you see a comparison of long term versus immediate bias impact, another fascinating result.

I am sure I am making this too simple, but it seems like a start.  Shoot me down, please.

.