The New York Times Economix blog has an interesting post on outcome measurement.

The writer rebuts the skepticism among readers about the viability of quality measurement — including an aside about the justice area:

These readers appear to harbor genuine doubt that quality in health care can ever be properly defined and measured. But what is the alternative — just relying on anecdotes and word of mouth, or the assurances from health care providers that they provide the highest quality of health care in the world?  

It is, to be sure, challenging to measure the quality of any human-service sectors, be it health care, education, the administration of the law or even corporate management. That is why anecdotes and word of mouth remain important signals that attract or repel individuals from particular products or institutions.

But flight once seemed impossible, too, perhaps even after the Wright brothers’ first flight. “No flying machine will ever fly from New York to Paris,” Orville Wright famously said, because “no known motor can run at the requisite speed for four days without stopping.” Wright also offered the thought that “if we worked on the assumption that what is accepted as true really is true, then there would be little hope for advance.”

The large and growing cadre of clinicians and measurement scientists engaged in measuring quality in health care can find inspiration in aviation. They persist, and they have registered much more progress in recent decades than might be imagined — much more, for example, than has been achieved in other human-services sectors, notably education, not to mention what we call the administration of “justice.” (Bold added)

The blogger, Uwe Reinhartd of Princetown, goes on to analyze the multiple complexities of inputs and outcomes, in an analysis that is strikingly similar to our discussions about courtroom outcomes as being insufficient to measure justice outcomes.

In the health care production process, quality can be monitored on several facets:

• The characteristics of the purchased inputs used in production of health care — e.g., the training of health personnel, the sophistication of the equipment supporting health professionals or the degree to which the architecture of facilities encourages or hinders patient-centered health care;

• The structure within which health care production takes place — e.g., the degree to which the production of health care is clinically integrated, including the electronic information technology that enhances or hinders that integration;

• The treatment processes for particular medical conditions — e.g., degree of adherence to known best clinical practices (expressed in practice guidelines and clinical pathways derived from these guidelines), processes that avoid hospital-generated infections and avoid re-admissions that could have been avoided, and so on;

• The impact of medical interventions on the patients’ health and well-being in the short and long run, often referred to simply as “outcomes” — e.g., survival rates by time periods, functional status, pain and so on;

• And, very important, satisfaction of patients with the treatment processes they have experienced, measured by means of surveys, ideally not administered by providers themselves.

This particular division of quality metrics goes back to a classic paper on the quality of health care published in 1966 by Dr. Avedis Donabedian, a distinguished physician and a towering figure in the field of quality measurement who died in 2000.

The post concludes by urging the development of multiple outcome scoring systems:

Efforts to hold health care providers formally accountable for the quality of their care are rarely one-metric systems. Instead, they resemble a final examination in a college course, with scores on many different questions, each with a relative weight, which are then totaled as a weighted sum to produce the final overall grade.

There is much to learn from here.  We have much to learn from our medical siblings. The post has some charts that are very helpful in understanding the complexities.