This blog is proud to be interviewing Professor Jim Greiner of Harvard about his latest research, conducted with Cassandra Wolos Pattanayak and Jonathan Hennessey, into the impact of offers of representation on outcomes.  The research is summarized in a recent post, and I will not repeat the summary here, except to say at the grossest level that one study showed very dramatic impacts from offers of representation, and another, applying the same law, but in a different court, representation and client pool selection environment, showed little if any.

The research, as always, raises as many questions as it answers, so lets get going, starting with the obvious major potential conclusions.

Zorza:  Jim, the research, particularly the study conducted in a Massachusetts District Court, is obviously very encouraging for advocates of a dramatically increased role for counsel in eviction cases, in showing a very major impact from offers of such representation relative to more limited services.  You warn in your paper about over- and under-generalization.  Are there caveats or reasons to be skeptical about such a positive result?

Greiner:  First, from the perspective of those who hope that the court system can provide access to all, I find the District Court Study results worrying.  They suggest that despite the best efforts of a judge with a legal services background, and despite the efforts of dedicated court staff, there may well be a severe access to justice problem in this court’s summary eviction calendar, at least for the set of potential clients who were brought in through the service provider’s outreach and intake system during the study, and who do not have counsel.  This is because we found such dramatically different outcomes when people were offered representation (a situation we would hypothesize would be most likely to lead to the result most in accordance with the law, given that the other side was usually represented.)

There are lots of possible ways to overgeneralize the District Court Study findings.  Building on our experiences with the Harvard Legal Aid Bureau unemployment study, we continue to hypothesize that legal services provider intake systems make a major difference in the type of potential client, the case circumstances, and the range of legal circumstances that a provider sees.  In the District Court Study, the provider (Greater Boston Legal Services, or “GBLS”) expended substantial resources doing outreach as well as investigating and screening cases.  The result was that our dataset consisted of cases in which, for 70% of the potential clients, the legal services provider had taken the first step in initiating contact with them.  That means that the set of clients in the study is NOT limited to those with sufficient self-motivation, access to social networks, courage, etc. to find and initiative contact with a provider.  But it is also true that GBLS expended substantial resources in intake and screening, so that our dataset consisted of cases in which GBLS guessed that it could make a difference in the outcome.  We speculate that we would not have seen such dramatic differences between the District Court Study treated and control groups had GBLS not done this outreach and screening.

Zorza:  And, what might be the ways of seeing the impact as more general than might at first seem obvious?

Greiner:  One primary way in which the findings of the District Court Study could be (wrongly) undergeneralized would be for other courts or court systems to say, “Well, that doesn’t apply to me because of [fill in some difference between the District Court in this study and my court].”  The fact is, we still don’t have enough rigorous studies to be able to say what court characteristics, defendant characteristics, housing stocks, state laws, etc. lead to access to justice problems.  So court systems cannot assume that the District Court Study findings would not apply to them.

In terms of non-obvious ways in which the District Court Study might be more general than might first appear, one factor might be the legal services background of the District Court Study judge.  One might believe that a judge without such a background would have been less willing to assist the control group defendants in the District Court Study, 89% of which were forced to self-represent (after receiving help in an instructional clinic).  If so, then in another court, with a judge or set of judges without legal aid backgrounds, an offer of full representation might cause an even greater difference between treated and control group outcomes that what we saw in the District Court Study.

Zorza:  Similarly, the study conducted in a Massachusetts Housing Court seems potentially very encouraging for those who think that attorney-of-the-day services, at least in an environment of an engaged court, can provide equivalent benefits to those provided by traditional “full representation”.  Are there any caveats or reasons to be skeptical about this result?  Any reasons to see it as particularly generalizable? If so, in what directions?

Greiner:  As we say in the Housing Court Study paper, for several reasons, we remain unable to credit fully the assertion that Lawyer for the Day (“LFTD”) program run by the legal services provider (Neighborhood Legal Services, or “NLS”) succeeded in duplicating the outcomes that a vigorous program of offers of full representation should have been able to produce.

First, the outcomes for both the Housing Court Study treated (offered full rep by an NLS staff attorney) and control (referred to NLS’s LFTD program) groups look a lot like those experienced by the District Court Study control group.  Now, there are lots of reasons why a comparison of outcomes between the Housing Court and District Court Studies is dangerous and speculative.  Among these reasons are that the two studies took place in different courts, with different sets of clients, in different geographic areas (and thus different housing stocks).  So we’re guessing here.  Nevertheless, if one is willing to engage in this kind of guesswork, we find it worrisome that the outcomes experienced by the Housing Court Study treated and control groups both looked so similar to the outcomes experienced by the District Court Study control group.  If the story is that the LFTD program duplicated the results of a rigorous program of offers of full representation, then we would have expected to see the outcomes experienced by the Housing Court Study treated and control groups to look more like the District Court Study’s treated group.  We didn’t see that.

Second, only slightly more than half of the control group occupants, all of whom were referred to the LFTD program, actually did use it.  In contrast, around 82% of treated group occupants, who were offered full representation, actually used full representation (the percentage was higher for occupants whose cases went to litigation).  We would have been more comfortable with the equation LFTD = full rep if the percentage of eligible occupants who used LFTD had been closer to the percentage of eligible occupants who used full rep.  As it is, it seems as though something else, or at least something more, might have been going on.

Third, there was little difference between the Housing Court Study treated and control groups in the rate at which jury trials were demanded or in the number of prejudgment motions that occupants filed per case (this even though the LFTD program did not provide assistance in filing motions).  Further, both the jury trial demand rates and the number of prejudgment motions per case in the Housing Court Study treated and control groups were far lower than the corresponding rates in the District Court Study.  From speaking with housing attorneys, we understand that jury trial demands and pre-trial motions, particularly motions to compel responses to discovery, are important tools for attorneys defending summary eviction cases.  These tools seem to have been used with less frequency than we might have expected in the Housing Court Study treated group.

Zorza:  Some have suggested that this is not really “Civil Gideon” research, but rather triage system research, because you are really testing the impact of comparative outcomes of different intake and triage systems, in one of which the only cases allowed in the pool were those that attorneys thought that counsel would make a difference.  Looking at the two research projects this way, what conclusions might you draw?

Greiner:  I guess this depends on how one defines “Civil Gideon.”  For us, little turns on how one defines the term.  In our view, the important point is figuring out how to assure full access to justice for all persons who need it.  Personally, I would be satisfied with assuring a decent minimum of access to all persons who need it.  I think we’re so far from either goal (full access or a decent minimum) that there’s enough research work to do for anyone who has the slightest interest in research, regardless of one’s preferences on how to tackle the challenge from a policy point of view, or how whether a particular branch of research deserves the brand name “Civil Gideon.”

Russell Engler has persuaded me that the right way to think about all this is to view a right to counsel within the broader context of an access to justice-promoting agenda.  And you have persuaded me that the Supreme Court seemed to view things this way in Turner.  Of course, the Turner Court was blissfully relieved of the annoyance of  considering evidence regarding the access to justice questions the Turner facts implicated, such as (i) what sort of legal issues that regularly arise in child support proceedings in the states, and the level of complexity and difficulty of those legal issues, (ii) whether the vaunted by entirely hypothetical “form” in which the Court placed such faith would actually serve its intended purpose of eliciting the necessary facts from a party given no assistance in filling it out, and (iii) how states would respond fiscally to a holding that counsel was required in child support contempt proceedings in which a party may end up incarcerated.  Nevertheless, relieved of the unfortunate messiness that actual facts might have imposed on its analysis, the Court was clear that counsel is one among several access-to-justice-promoting and due-process-promoting measures that might be necessary in an individual cases or in particular classes of cases.  From a top-level theory point of view, in living in a world of constrained resources, that seems right to me.

Regarding resources, my personal view is that there we will never come anywhere close to getting enough funding to provide an offer of full representation to persons with (civil) legal problems.  Perhaps more controversially, my personal view is that we will never come close to enough funding to offering full representation to all persons facing adjudicatory proceedings that implicate basic humans needs (per the recent ABA resolution.   If I’m right about either assertion, then everything has to be on the table:  simplification of and alterations to substantive law; reform of court practices, triaging levels and types of legal assistance; etc.  Channeling Russell Engler again, my suspicion is that there will be a class of cases in which nothing short of an offer of full representation will be enough to assure minimally decent access to justice.  But unchallelling Russell Engler, whether we can identify cases that fit into that class early enough in the proceedings on the basis of a record presumably developed without the assistance of counsel is a big question.  In fact, what works and what doesn’t work on all of these fronts constitute big questions.  Again, there’s enough research work to go around.  Folks should feel free to use whatever labels, civil Gideon or otherwise, they feel comfortable with for this research or any other in which we engage.

Zorza:  Following up, what kinds of research would clarify his picture more?  Would it make sense, in the same court, to treat two groups differently in terms of how they were triaged/recruited, and then split each of the groups in term of representation?

Greiner:  In short, yes, with a slight clarification of terms.  One idea would be to take a source of possible potential clients (such as the set of defendants against whom summary eviction actions were filed in a particular week) and randomize them to different outreach and intake systems.  For example, one could randomly assign half of a set of defendants to a letter from a service provider inviting them to an instructional clinic (as was done in the District Court Study) and the other half to a personal visit from a law student thoroughly trained to conduct an assessment interview at the occupant’s unit (note that law and other students are going to summary eviction defendant’s homes in Project No One Leaves.  One could then randomize within each of the two groups to different levels of legal assistance (e.g., a referral to a LFTD program versus an offer of full representation).  A study design like this one would require probably require either a high volume court or a fair amount of time, and it would have to be well-resourced.  But this kind of study could tell us a huge amount about the effect of different outreach/intake/screening procedures, the effect of different levels of representation, and the combination of the two.

Zorza:  Does this, for example, support the theory that legal aid programs are the best equipped to do triage?  How would one test this theory?  Is this civil Gideon?

Greiner:  To my knowledge, no study, including ours, has tested rigorously who can best or most-cost-effectively make triage decisions.  Suppose law students (free labor!) could be trained somewhat cheaply to make reasonably accurate triage decisions, while legal aid staff could do so faster and more accurately but at the cost of tying up their time, which could otherwise be spent providing services.  What’s the right way to do it?  We don’t know, but we’re pretty sure that it would be a good idea to know the relative efficiency of each type of potential triager.

Note that, again, one way to interpret the huge treated versus control group differences in the District Court Study is that legal aid providers can identify a set of cases in which an offer of full representation will make a large difference, but they may not be able to do so cheaply.  If it’s true (and we didn’t test it directly), that’s nice to know, but it’s so far from what we need to know to decide who should do the triaging, and how those chosen should go about triaging, that we have a lot of work to do.

Again, this kind of thing can be studied rigorously.  At the risk of beating a dead horse, one could do a solid randomized study by randomly assigning potential clients to one of two (or more) possible triage systems.  Then follow the results.  See which triage system is the most effective, using several different measures of effectiveness.

Zorza:  I note that in the District Court study, the attorneys were highly experienced advocates who, I suspect, believed very strongly in the value of their work.  Could this have increased the treatment effect?  If so, it only underlines how hard it is to do this research in a clean way.  Is there a legal aid placebo?

Greiner:  The problem isn’t so much a placebo as it is multiple levels/forms of active treatment.  One could probably analogize a placebo to assignment to no offer of any form of study-related legal assistance, not even limited or unbundled legal assistance.  Depending on the situation, the use of this kind of placebo in the legal context could raise ethical issues (that’s also true in medical studies).  But the question you ask does point to a closely related problem:  there might be multiple forms of treatment, analogous to multiple dose levels of a drug.  A high dose might be an uber-laywer, while a less high dose might be a solid-but-not-superstar lawyer.  We speculate that we were dealing with uber-lawyers in the District Court Study, but we must acknowledge that we have no agreed-upon, objective measurement for strength of lawyer.

It’s worth noting here that a randomized study in criminal law suggested that criminal outcomes varied greatly depending on which of several public defenders was assigned to the case.  This study also found that one of the best predictors of whether a particular attorney would achieve favorable results was the number of years that attorney had practiced.  But number of years of practice won’t fully explain our results:  one of the attorneys in our Housing Court Study (which showed no effect of an offer of representation) had 25 years of litigation experience (although only a couple of years as a housing litigator).

Again, this is an area where additional information-gathering, much of it non-statistical, could be extremely helpful.  In Stapleton & Teitelbaum’s study in juvenile delinquency proceedings, the attorneys in the study wrote short reports about what happened in each of their cases.  Those reports allowed the researchers to explain the results they saw much more completely than we are able to in our studies (at least thus far).  In fact, at least thus far, we have been unable to get accurate information on a matter as simple as the number of hours expended per case in both the treated and control groups.  You might ask:  Why were Stapleton and Teitelbaum able to do what we could not?

The answer is that when we began this research project, we lacked the street cred with legal services providers to suggest much of anything in the way of solid research methods.  We put all the political capital we had into getting those involved to randomize.  Some were initially reluctant, but to their tremendous credit, they eventually agreed.  But  we were brought in very, very late into the planning process for these two studies, after the funding had been raised, the sites selected, the funding recipients identified, the nature of the representation and the relevant attorneys fixed, the time period for the studies chosen, the budget for the evaluation (which was $0.00) set, and the project set to begin in a few months.  At the end of the planning process, we had no political capital left for anything else.  We hope that the publication of the District and Housing Court Studies will induce folks contemplating access to justice research in the future to contact nerds earlier in the process (meaning at the very beginning).

One final, potentially critically important point;  If in fact there are major differences between the effectiveness of individual attorneys, such that some attorneys make little or no difference and some make a huge difference, then the story for legal aid providers becomes very complicated.  The best story for the legal aid community is:  “We can make big differences for clients, we have the right horses already in place to get the job done.”  If the right horses may not be in place, the story is less clean.  We are not saying that we know that this is what happened in the District or Housing Court Studies.  We’re saying that if this is part of the story, the implications are complicated for legal aid providers, but also very important to know.

Zorza:  Any thoughts from this research about the consequences of legal complexity upon the impact of counsel? About the consequences of law that provides broader rather than narrower rights to tenants?

Greiner:  Many poverty lawyers have pointed to summary eviction law as among the most complex substantive legal areas in which they practice.  And the District Court Study suggests that offers of full representation from traditional legal aid staff attorneys can have a large impact on outcomes.  Looking beyond the numbers, my review of the cases files in the District Court Study suggests that the many of the defenses and counterclaims GBLS raised were factually and legally complex.  They involve combinations of state statutory law, state common law, and federal regulations, and they required comparatively extensive factual development.  One example here is incorrect calculation of tenant rent contributions under the Federally funded Section 8 housing voucher program.  Another is the presence of lead paint or other possibly illegal contaminants in the unit.  A third is cross-metering of utilities.  Again, the source of law varies here, in addition to its content, and pre-hearing factual development is a must to assert these defenses adequately.

On the other hand, the lack of large effects in the Housing Court Study suggests legal complexity alone cannot predict whether an offer of full representation (as opposed to a more limited intervention) will make a difference.  Remember that one the things that was absolutely the same in the District Court and Housing Court Studies was the applicable law (at least the law-as-written; it’s possible that the law-in-practice might have varied).

My gut instinct is that legal complexity has an important role to place in explaining why, and predicting when, legal interventions may change outcomes.  But my gut instinct is that this factor does not deserve the enormous focus it has received from many folks with whom I talk.  For me, what has been undersold are the client characteristics (self-motivation, articulateness, access to social networks), and the corresponding implications of variation in those characteristics for outreach, intake, and screening systems.  Few people with whom I speak instinctively think of these factors.

Zorza:  At one point you hypothesize that we can assume that the fairest result comes from when both sides are represented by competent counsel, and that other systems should be compared to this.  Doesn’t this create a possible circularity in the research?  How would one test that hypothesis?  Are there other definitions of optimum outcomes?

Greiner:  Actually, one can’t really test this hypothesis.  For me, this is an example of having to live with a third-best or fourth-best measuring device.

Stepping back, the basic question is, how do we measure whether a court or administrative system is reaching the “right” results in its run-of-the-mill cases?   “Right” here is in quotation marks because it assumes, contrary to those who hold that truth and right answers in law are illusory or are constructed (see the movie Rashomon, which concerns a criminal trial), that there is an adjudicatory output that represents the proper application of reasonably knowable law to reasonably objective facts.  Assuming that all this is true, how do we know whether a court is reaching the “right” results?  Obviously, if one is trying to measure whether Court A is reaching the right results through its normal operations, one can’t very well use the results of Court A’ s run-of-the-mill operations.

One way to tackle this issue is to create a second, separate, super-court.  In other words, if we’re trying to measure whether Court A is reaching the “right” results, we create a Court A+.  Give this Court A+ “superjudges,” a super investigatory capacity to develop facts, superstar well-funded lawyers to find and develop the law, etc.  Then send cases through both Court A and Court A+.  See how often the decisions/outputs of Court A match those of Court A+.  This doesn’t strike me as very practicable.

Our alternative idea in the paper was to take a court in which one set of potential clients, (such as summary eviction defendants) is essentially unable to obtain legal representation, but where parties opposing this set of potential clients (summary eviction plaintiffs) usually are represented.  Randomly divide the set of potential clients who are usually unable to find representation into two groups:  in the treated group, offer full representation from attorneys funded by some grant.  In the control group, offer something less than full representation.  In the treated group, if most offerrees accept the offer of representation, then both sides will end up represented.  In the control group, only one side is represented, which is the ordinary way that this particular court functions.

Now comes the big assumption:  suppose we assume that when both parties are represented by competent counsel, the adversarial process will cause the “truth” to emerge and the “right” result to be obtained.  This is an enormous assumption, and it may not be true.  It might depend on how competent the various attorneys involved are, for example.  But under this assumption, that the “right” answer emerges from an adversarial process which both parties are represented, is the fundamental assumption upon which our court system rests.  And if we’re willing to make this assumption, then the treated group in the previous paragraph should provide a good idea of what the “right” answer is in a set of cases, while the control group in the previous paragraph provides a picture of how the court usually operates.  If the outcomes for the treated and control groups differ systematically, then under the big assumption just articulated, we have evidence that the court’s adjudicatory system is not regularly producing the “right” results.

Note that saying that a court’s adjudicatory system is not regularly producing the “right” results does NOT mean that the judges or clerks or anyone else in the court staff is doing anything wrong.  There are plenty of reasons, such as the complication in the area of law or the need for factual preparation, that adjudicatory systems might be unable to reach the “right” results despite the court staff’s best efforts.

We think all of this happened in the District Court Study.  There, about 90% of would-be evictors (usually landlords) were represented, but (absent this pilot effort) almost no defendants (usually tenants) were.  As a result of the Study, 92% of the treated group ended up represented, while only 8% of the control group did.  Under the assumption articulated above, the results for the treated group are a good proxy for the “right” results, while those of the control group provide a good idea of how the District Court operates regularly (at least with respect to the set of clients that were eligible for the Study).  The results for the treated group, the “right” results, were very, very different from those in the control group.  That suggests a severe access to justice problem in this District Court, at least with respect to the set of clients identified by GBLS’ outreach, intake, and screening system.  Again, all of this was despite the best efforts of the judge and the court personnel involved.

Zorza:  If you were a court administrator, what message would you take from this research, and what would you be seeing to do with this information?

Greiner:  That despite my best efforts, there could very well be a set of cases out there in which my adjudicatory processes are insufficient to assure a decent minimum of access to justice, and that for this set, offering full representation to the relevant clients in these cases may be enough to assure a decent minimum of access.  (It may or may not be possible to assure a decent minimum of access by offering a form of assistance short of full representation; for reasons explained above, we don’t view the Housing Court Study as conclusive on that point.)  Again, this could be despite my best efforts as a court administrator.

Zorza:  If you were a legal aid project director, what would you be doing?

Greiner:  Taking a very, very, very hard look at my outreach, intake, and screening systems.  As I did so, I’d keep firmly in mind that if my organization depends on client-initiated contact, and that if we conduct intake interviews over the phone, we could be culling clients.  In other words, those that can reach us may well be those that need us least.  I would also think about whether I am on notice about the importance of assessing the quality of the representation we provide, because it may indeed make a huge difference to outcomes.

Zorza: If you were an Access to Justice Commission member, how would you be thinking of using this, both politically and programmatically?

Greiner:  Programmatically, I would try to persuade legal services providers (meaning traditional legal aid providers, pro bono programs, law school clinics, etc.) to engage in rigorous, preferably randomized, evaluations of multiple aspects of their own programs.  I’d focus particularly on intake systems as well as triage systems.  We need to know so much more than we do know right now about how intake and triage actually work.

Similarly, I’d try to persuade courts to engage in rigorous, preferably randomized, evaluations of court-based access to justice promoting measures.

Politically, the choices are more difficult.  Among the things I’ve learned in the past year is that I lack the political sense, and the familiarity with the power brokers in the access to justice debate (the legal aid providers, the judges, the court administrators, the organized bar, others), to make political recommendations.  I do now know what I did not know a year ago:  that rigorous research (at least in this area) is certain to anger someone.  The unemployment study angered some legal aid providers; the Housing Court Study has done the same.  The District Court Study angered at least one judicial employee that I know of; third-hand information suggests that there are others who feel similarly.  Our research program here has recently been compared (implicitly and otherwise) to the efforts of the Gingrich Congress vis-à-vis the Legal Services Corporation.  This and others like it have forcefully reminded that I do not have the savvy to make political recommendations.

Zorza:  If you were trying to design a triage system, where would you go now?

Greiner:  I’d do more research on triage systems, as suggested above.  But until the research provides more guidance, I’d recognize that the information needed to make educated guesses on triage decisions may be impossible to obtain on the cheap.  In person interviews, limited investigation into underlying facts, and perhaps even some limited use of discovery devices, all might be necessary to make intelligent triage decisions.  The investment of resources into information-gathering will be worth it if the result is intelligent and intentional allocation of cases to appropriate interventions.

Zorza:  What research do you have coming down the pipeline now?

Greiner:  We have two other randomized studies ongoing, one in SSI/SSDI benefits, the other in divorce.  The SSI/SSDI study compares an offer of representation from a student-based clinic to the provision of a list of other possible legal assistance providers who handle such cases.  The divorce study compares the offer of a private, pro bono lawyer to the availability of telephone advice from a highly knowledgeable family law attorney.

We also have a randomized study of the effect of an order for the parties to engage in mandatory arbitration, as opposed to an order encouraging the parties to engage in informal settlement negotiations.  The subject area here is state prisoner’s civil rights complaints.  Proponents of mediation programs justify them with the argument that the parties in mediated cases are more subjectively satisfied with the resolutions they achieve.  In New York, mediation is being justified in foreclosure cases in part on access to justice grounds.  Perhaps unsurprisingly, no one to our knowledge has assessed rigorously whether mediation actually accomplishes any of these goals.  Note that a vanishingly small percentage of the prisoner-plaintiffs in our study have access to counsel, so the access to justice implications of our study are evident.

Zorza:  In one of the papers you talk about the need for research into the impact of the deployment of judicial best practices for the self-represented.  How would you see such research being organized?

Greiner:  The best thing to do would be to randomize the practices, then measure outcomes via looking at court files and via surveying litigants.  Depending on the practice, this could be hard to do.  If the practice is, for example, providing an en masse explanation to a group of litigants whose cases are on call for a particular day of the nature of the proceedings and the kinds of evidence that would be useful for the case, then randomizing at the case level could be difficult.

But a variety of other practices can, and should be, randomized.  We could train judges previously accustomed to passive adjudication to be able to employ active and engaged questioning at hearings, then randomize the cases in which the judges use the engaged style versus the passive style.  There’s a lot of rigorous work that can be done here with creativity on the part of the nerd and willpower on the part of the judges.

Zorza:  What other research would you like to be doing, but do not yet have in place?  How should people be in touch if they think that they might have a test bed for such research?

Greiner:  There’s a huge variety of research I’d like to pursue.  Here are three priority categories for me.  First, I would like to expand the range of outcomes measured beyond adjudicatory outputs to (i) satisfaction surveys of potential clients (such as what we’re already doing in the mediation study), and (ii) socioeconomic indicators of well-being, such as credit scores.  Second, I would like to study triage systems, as articulated above.  Third, and perhaps most importantly, I would like to study whatever those in this field would find useful to have studied.  We need more in the way of communication between power brokers in this area (legal aid providers, judges, court staff, the organized bar) and researchers.  I would very much like to engage in that dialogue.

I have only two conditions for participating:  first, I won’t participate if the fix is in.  What that means is, if some says, “I need a study that supports my argument that X is a good idea,” regardless of what X is (it might be offering full representation from a service provider attorney, it might be a mediation program), there is no point in engaging in the research.  The fix is in.  Second, I publish the results of any study in which I’m involved.  I do not subscribe the view that open access to rigorous information is too dangerous for this field.  This view reminds me of one of my favorite (meaning sickness-inducing) quotes; it comes from General William Westmoreland:  “Without censorship, things can get terribly confused in the public mind.”  Change “Without censorship” to “With accurate information” and . . .

For folks who think these conditions are OK, please, please get in touch!!  Email (jgreiner@law.harvard.edu), telephone ((617) 496-4643), post-owl, whatever works.  The research community, including me, needs further guidance on what would be of use in this area, and we need partners who are willing to engage.

Zorza:  By the way, in these studies, the Boston Bar, and the Mass and Boston Bar Foundations provided money for extra lawyers who then provided both the control (limited) and the treatment (full) representation.  This is a pretty expensive way of doing research if you have to add services, rather than research existing services.  Is there any ethical barrier to putting a randomization step into an existing overloaded delivery system, and then not having to pay for extra legal services.  Any other barrier?

Greiner:  The only major barrier is will.  Had my coauthors and I been brought in early in the set of steps that eventually led to the District and Housing Court Studies, we would have suggested that for the amount of money invested in these studies, we could have done more than two rigorous evaluations of existing legal aid programs.  These more numerous evaluations could have included more statistical and non-statistical information-gathering, so that we would know so much more than we know now as a result of the research and would have fewer unanswered questions.  Again, for a randomized study that did a whole lot right in this area from a research standpoint, see Stapleton & Teitelbaum’s study in juvenile delinquency proceedings.

One has to decide what the primary goal of a funding stream is.  Right now, money allocated to evaluation and research is viewed as money taken away from service providers and courts.  If a funding stream is primarily there to fund services, regardless of whether those services work, or might be made to work better (however one defines “work”), then that’s the right view.  If a funding stream is in part there to gain information about what works and what doesn’t, and how things might be made to work better, with an eye to an eventual (and far more powerful) argument for increased funding, then that view is dangerously wrong.

Zorza:  You have now done three different randomized studies on the impact of offers of counsel on outcomes.  Have you come to any conclusions on the viability of such research in the access to justice context?

Greiner:  It works.

Zorza:  Thanks again for doing this interview, and congratulations on some very important research.  We very much look forward to the next round.