This blog posting inaugurates a new feature, series blogs. When I think an topic is particularly important, I will blog several times about the topic. Each series will get its own listing in categories, so you can quickly find the whole sequence.
I am starting with a series on outcome measures because they remain somewhat controversial within the access to justice community, and I think it is important to lay out where we are and where we can and should be going.
Let me start by going back to the medical analogy. Those of us who are thinking about treatments and drugs, and potentially life-changing choices constantly worry about things like impact on life expectancy. While we may quibble (with energy) about the importance of designing quality of life outcome measures that value more than just time gained — and indeed I am enrolling in just such a study to validate such measures with respect to my disease — the fact is that we would be lost without that data on outcomes. Imagine if we had to rely on an oncologist saying “Oh, we think that we have found from experience that this drug is better than that,” or “we tell you to hope for the best and prepare for the worst.” (One of my favorite lines from Terms of Endearment is when the Shirley Mclaine character, in response to a doctor saying that that is what he tells all his patients, she shoots back, “And they let you get away with it.”)
But without validated and legitimate outcome measures that’s exactly where we are in the legal system. And, just as it would be hard for the medical system to get much research money if they had not committed to studies based on outcome measures a few years before they they went for the big federal money, we are going to find it very difficult to get innovation and research money unless we can show that we are learning from the investments we make. Indeed, while some would worry that it is unethical to do randomized experiments, I would argue that it may be more unethical to continue to “treat” people’s legal problems if we refuse to use available techniques to get data from which we could make better choices about how to do so.
But even at the substantive level, how can we even argue for innovation if it will not teach us anything? How can we propose simplification, for example, if we have no way of looking at what the impact of any changes are? How can we design incentives for simplification if we have no way of measuring what the benefit is?
How, similarly, can be we make decisions as to who would need and benefit from what services, unless we have a way of measuring the impact on outcomes of different services in different situations? Without outcome measures, any triage systems will be intuitive at best, and surely inconsistent in their application.
Perhaps most importantly in this moment of unqie opportunity, how can we develop and test strategies for 100% access without the “realistic and measurable outcomes” urged by the CCJ/COSCA Resolution.
So we, and by “we” I mean both courts and advocacy organizations, have to get over this hump and design accepted, legitimate, and validated outcome measures.
I would note that one of the hesitations about doing so comes from the continuing confusion about the relationship between what courts need from such measures and what representation/advocacy organizations need. The two sets of needs have to be different, because advocates want to get the best possible result for each of their clients, while courts want to be fair, neutral, and accessible to all. I have come to believe that it is possible to develop an overall system of outcome measures that meets the goal of reflecting these different needs and perspectives, and yet ultimately allows the two kinds of institutions to be assessed as an integrated whole.