In a major development for access to justice, the next round of studies into the impact of offers of counsel, this one conducted by Jim Greiner, Cassandra Wolos Pattanayak, and Jonathan Hennessey, is now available.  They are likely to once again transform the debate on the relationship of offers of counsel to access to justice.  The studies will be the subject of a forthcoming NewsMaker Interview on this blog.

In super short summary, one of the two studies, conducted in a District Court in Massachusetts, (link is to Social Science Research Network) compared the impact of randomized offers of representation in eviction cases on a pool of defendants recruited by the legal aid service provider, to a control group that no such offer.  Members of both groups were offered, and most received, assistance with answer and discovery forms and an instructional lecture.  The study found dramatic impact on eviction outcomes and financial outcomes, without significant burden on the court.

However, the second study (now linked to SSRN post) yielded a superficially surprisingly different result.   In this second study, conducted in a Massachusetts Housing Court, (specializing in housing cases only, in this case covering a different area) no significant such outcome differences were found in comparing similarly randomized offers of representation to a pool of individuals who sought the program’s help.  In this study the control group received a referral for “attorney of the day” assistance in mediation and negotiation, but not in the courtroom or in the filing of motions, and the court had a very engaged program of mediation, in which mediators investigated and made outcome predictions to the parties.  Both treatment and control groups were offered and most received forms and discovery assistance.

Here is the abstract of the first (District Court) study:

We persuaded entities conducting two civil Gideon pilot programs to randomize which potential clients would receive offers of traditional attorney-client relationships from professional service provider staff attorneys and which would receive only limited (“unbundled”) assistance. In both pilot programs potential clients were defendants in housing eviction proceedings, and both programs were oversubscribed.   In this Article, we report the results of one of these two resulting randomized control trials, which we label the “District Court Study,” after the type of the court in which it took place. In the District Court Study, almost all study- eligible eviction defendants received limited assistance in the form of help in filling out answer and discovery request forms, and most also attended an instructional session on the summary eviction process. After receiving this limited assistance, each member of a randomly selected treated group received an offer of a traditional attorney-client relationship from one of the legal services provider’s staff attorneys; each member of the corresponding randomly selected control group received no such offer. We compare outcomes for the treated (offered traditional representation from a service provider staff attorney) group versus the control (no such offer) group on a variety of dimensions, focusing primarily on possession of the unit, financial consequences of the litigation, and measures of court burden.

At least for the clientele involved in this District Court Study, a clientele recruited and chosen by the service provider’s proactive, timely, specific, and selective outreach and intake system, an offer of full representation mattered. Approximately two thirds of defendants in the treated group, versus about one-third of defendants in the control group, retained possession of their units at the end of litigation. Using a highly conservative proxy for financial consequences, treated group defendants received payments or rent waivers worth a net of 9.4 months of rent per case, versus 1.9 months of rent per case in the control group. Both results were statistically.  Meanwhile, although treated cases did take longer to reach judgment, the offer of representation caused no increase in court burden as measured by other, more salient metrics.

Our results are interesting on a different dimension. A fundamental assumption of the adversary system is that the “right” answer will emerge from a process of contested facts and law in which both parties are represented by competent counsel. In our treated group, 86% of plaintiffs and 97% of defendants were represented. Under the aforementioned assumption, then, the outcomes in our treated group are a strong proxy for the “right” results in summary eviction cases, at least with respect to the class of potential clients involved in the study. The disparity in outcomes between our treated and control groups suggests that, with respect to the clientele in this study, the District Court summary eviction process did not produce the right results for control group defendants. This was true even though control group defendants received substantial (but limited) legal assistance, and even though the adjudicatory process included certain measures designed to promote access to justice, such as mediation and some judge-initiated questioning. Thus, the adjudicatory system did not provide full access to justice despite the best efforts of personnel within it.

We discuss possible reasons for the magnitude of the differences between our treated and control groups. In particular, we highlight that our randomized design allows gold-standard inferences about how much an offer of full representation matters for potential clients who had already received substantial legal assistance. But we also highlight that our results may suggest that isolating a set of clients for whom limited representation is inadequate may require service provider investment in a proactive, timely, specific, and selective outreach and intake system. For these and other reasons, we caution against either overinterpretation and underinterpretation of this study.

After reporting the results of the District Court Study, we offer thoughts as to the future of the study of the limited legal assistance programs and of legal services programs in general.

Here is the abstract of the second (Housing Court) study:

 We persuaded entities conducting two civil Gideon pilot programs to randomize which potential clients would receive offers of traditional attorney-client relationships from professional service provider staff attorneys and which would receive referrals to only limited (“unbundled”) assistance. In both pilot programs potential clients were occupants in housing eviction proceedings, and both programs were oversubscribed. In this article, we report the results of one of the two resulting randomized control trials, which we label the “Housing Court Study,” after the type of the court in which it took place. In the Housing Court Study, all study-eligible potential clients could (and most did) receive assistance in filling out answers and discovery requests. In addition, occupants not offered a traditional attorney-client relationship from the provider’s staff attorneys, i.e., the control group, received a referral to that provider’s lawyer for the day program. The lawyer for the day program provided same-day-only representation in hallway settlement negotiations and mediation sessions but not in court appearances or in filing motions. We analyze the intention-to-treat effect, meaning we focus on the effect of the provider’s offer of full representation vis-`a-vis the referral to the lawyer for the day program. In doing so, we evaluate the relative effectiveness of two types of programs a legal services provider might adopt.

We find no statistically significant evidence that the service provider’s offer of full, as opposed to limited, representation had a large (or any) effect on the likelihood that the occupant would retain possession, on the financial consequences of the case, on involvement in or attention to cases, or on any other litigation-related outcome of substantive import. To the contrary, treated and control group point estimates are close to one another. In addition, because about half of the potential clients in this program had contacted the service provider before litigation had been initiated, we are able to study whether the provider’s offer of full representation kept disputes out of court, although the limited size of the dataset made possible detection of only very large effects. We find no evidence of such very large effects.

We explore several possible interpretations of our results, and we caution against both over-interpretation and under-interpretation.

What on earth does all this mean?  This blog is proud to announce that in a few days Prof. Greiner will be participating in a NewsMaker Interview to explore all the implications of this important research.  We invite readers to submit questions for possible submission to Jim, either by comment on this post, or by e-mail to me at richard@zorza.net.  To get automatic e-mail notification of posts, subscribe to the blog using the tool in the right menu.

To help launch what will surely be a lively and ongoing debate, there are some thoughts.

First three general conclusions that I feel very confident about drawing (maybe they are obvious).

 1.         There are situations in which providing full representation in civil cases provides for significantly better outcomes for those who have this assistance, compared to some forms of limited assistance.

 2.         There are other situations in which providing full representation in civil cases provides for little if any impact on outcomes for those who have this assistance, compared to some forms of limited assistance.

 3.         In some court and representation contexts, full representation provides no greater protection against arbitrariness than “attorney of the day” programs.  In other such contexts full representation provides significantly greater protection than assistance with discovery and answers.

 In some ways, all that I am saying here is that I am completely methodologically persuaded by the papers.  These results cannot be explained in statistical artifact terms

Now, and more controversially, I offer a number of additional hypotheses that might help explain and clarify the results of the two studies.  The hypotheses are offered in approximately the order in which I find them intuitively convincing.  (This is all so speculative, that those next to each other in the order may well be identical in likelihood.)

• The court environment (taken as a whole) has a major impact both on overall outcomes and on the impact of the provision of counsel on outcomes.
• The representation provider has a major impact both on overall outcomes and on the impact of the provision of counsel on outcomes.
• Triage systems can work, at least in some contexts, identifying those who would benefit significantly from additional services (but we do not yet know if they are correctly selecting those that are most in need, that would require additional comparisons and controls.)
• Some legal aid programs are very good at identifying situations and potential clients in which there is likely to be benefit from additional representational services (but we do not yet know if they are correctly selecting those that are most in need).
• Because neither of these studies used the full court caseload as the pool, we do not know the impact of offers of full representation upon outcomes as a whole; all that was tested was two triage systems, one provider based, and one litigant selection based.  (I.e. strictly speaking this is not Civil Gideon research at all, it is provision of counsel research.)
• Courts that provide engaged investigation and outcome predication are more likely to experience smaller (or even zero) differences in outcomes when full representation is provided.
• Judges who do nothing but housing cases may well become impervious to the more aggressive claims that are advanced by lawyers providing full representation.  This may be true regardless of (or perhaps even because of) the prior non-judicial experience of the judge.
• Areas of law that are highly technical may require a greater skill and experience level before the impact of providing full representation is clear.
• While skilled advocates providing full representation can quickly move in and impact cases, it may take longer to establish limited scope services so that they are effective in a particular context.

These hypotheses could all be tested in the future.  As time goes by there will surely be opportunities to do so.

This is critically important work and all who participated are deserving of congratulations and thanks.  Particularly, of course, the Boston Bar Association Task Force on expanding the Civil Right to Counsel.  Here is their report, which obviously led to this research.  Without their support and funding, this would never have happened.

I look forward to the NewsMaker Interview, and hope you all do too.

Update History:

On October 27, I corrected the phrasing of one of the three initial hypotheses.  It originally read: “In some court and representation contexts, “attorney of the day” programs provide no greater protection against arbitrariness than full representation.”  As corrected, it reads: “In some court and representation contexts, full representation provides no greater protection against arbitrariness than “attorney of the day” programs.”    I hope the intended meaning was not too unclear in context.