David Udell has challenged me to identify, from the recent indicators report, some of the “best indicators.” Before I even think about doing that, I have tried to identify eleven criteria for a perfect indicator.
- The data is already being collected, even if not yet being processed and compared.
- There is a strong intuitive relationship between the indicator and the results/outcomes that we really care about.
- It is an indicator that will be sensitive to differences among subgroups under study.
- It it can be used to compare results in broadly different environments.
- It leads relatively easily to policy and management prescriptions.
- It is relatively cheap and easy to collect.
- It has broad legitimacy among multiple groups such as funders, providers, service recipients and the public, and it is not perceived as being too related to a particular institutional or other agenda.
- It is not too easily subject to being gamed.
- It is quantitative and subject to actual measurement and analysis.
- It aligns well with other actual or suggested measures, acting as a proxy for them.
- It, together only with a very small number of other indicators, provides a reliable overall view of the functioning of the system from all points of view and reflecting the needs of constituencies.
I would encourage those with really specific knowledge of their substantive areas to work together to identify five or so measures within that area that might do a better job of meeting those criteria, with the hope that together those five would meet the criteria in number 11.
Suggestions for improvements in this list would b much appreciated, including any ideas on how to get the list down to ten. (I assume that everyone knows the story about Moses coming down from Mount Sinai, and announcing: “First the good news, I got him down to ten. Now the bad news, adultery is still on the list.“)