A few days ago, my colleague Derek Muller and I posted a draft of a paper on SSRN entitled, The High Cost of Lowering the Bar. The paper analyzes a large dataset of lawyers in California and argues that the current proposal to lower the required passing score for the California Bar Exam would result in an increased rate of discipline and disbarment among the newly admitted lawyers with lower bar exam scores. We wrote the paper to provide an important consumer protection angle on the seemingly inexorable push by California law school deans to make the bar exam easier to pass.
The response to the study was almost immediate. The Wall Street Journal referenced our study in a larger article about the California Bar Exam and the Daily Journal wrote an article about our piece the next day. In the Daily Journal article, one of the most vociferous advocates for making the bar exam easier to pass, Dean David Faigman of UC Hastings, blasted our analysis as "about as irresponsible a product of empirical scholarship [he] could imagine for putting out for public consumption" and said, "God forbid anybody of policy authority should rely on that manuscript."
I thought that language was a bit intemperate for the dean of a law school, but it's standard operating procedure for Dean Faigman. After the July 2016 bar exam results were published, he attributed his school's performance to unspecified "shameful" and "unconscionable conduct" of the State Bar. Apparently what he meant by the shameful and unconscionable conduct was that the State Bar had administered the bar exam in exactly the same way as it always has since the 1980s. Dean Faigman likes to go public with over-the-top statements that are indefensible, so I can't take it too personally.
However, I do want Dean Faigman and others to understand that our analysis, far from being irresponsible, was actually an exercise of scholarly responsibility. An important public policy decision (lowering the bar exam passing score) is being evaluated right now, and the decision will affect the public for decades. My colleague and I identified a previously unnoticed risk associated with the proposal pushed by Dean Faigman and others, and called for the State Bar to analyze it. This is exactly what a responsible scholar would do given the very short amount of time that is left before a decision is made.
Because I do hope that Dean Faigman and others like him will come to agree that our analysis was an exercise of responsibility rather than a lack thereof, I have outlined below a few reasons why we published the piece when we did. I hope that Dean Faigman will agree that we have indeed acted responsibly in protecting the public interest. I especially hope that the State Bar will perform the data analysis we call for before acceding to the lobbying efforts of the California law deans who created the bar exam crisis in the first place.
First, the relationship between bar exam score and subsequent discipline that we identify is true, although we cannot identify the exact magnitude. Dean Faigman did not deny the core conclusion of our piece, that lawyers with lower bar exam scores are more likely to be disciplined. He attacked our methodology and data, but did not deny that the relationship we identify is accurate. Indeed, he would be unwise to make such a claim, as we are not the first to identify this connection. The very same thing was found in a Connecticut study sponsored by the Law School Admissions Council that did have all of the individual detailed data on the lawyers in the study. The authors of that study did not draw much attention to the bar exam-discipline connection, but it was there, buried "Model 4" of Table 8.
Second, about the lack of data that Dean Faigman criticizes... He's right that the publicly available data is not the best data to make precise estimates of the impact of lowering the bar exam passing score. Indeed, that was the whole point of our analysis--to call for the State Bar to analyze the data that we do not have access to. Our paper had one simple objective: to identify a risk associated with lowering the passing score and to encourage the State Bar to analyze the risk using its internal data. We say very clearly in the paper that "[t]he only way to make precise estimates of the impact of changing the passing score is for the California State Bar to use its internal records on bar examination scores and discipline outcomes to determine the likely fallout... Accordingly, we strongly recommend the State Bar undertake that analysis prior to changing the passing score." How is calling for an analysis of data unavailable to researchers irresponsible? Still, even the data we used, however, is more than enough to make the convincing claim for a connection between bar exam score and discipline.
Third, we did not have the option of accessing the State Bar's data. The State Bar is extremely reluctant to release data to researchers in general, in the past necessitating lawsuits to dislodge the information related to important issues of public policy. And even if the State Bar were willing to consider our request, there simply isn't time. Students are sitting for the bar in less than two months, and the deans are pressuring the State Bar to change the passing score for that test. If we had waited, the State Bar would have proceeded with its analysis without information about this important risk that (to the best of my knowledge) nobody else has identified in the context of this debate.
Finally, one of the concerns identified about our analysis by thoughtful scholars such as Deborah Merritt of Ohio State, is that low bar exam scores are not actually causing discipline, but rather merely correlated with it. The causal factors are likely others, such as the fact that lawyers who receive low scores on the bar exam also tend to come from less competitive law schools, have lower law school grades, and therefore end up in solo-practice settings or other environments where compliance with ethical rules is more challenging. I believe that Professor Merritt and other critics with similar concerns are (partially) correct about the causal arguments. It is not the failure to know Property and Torts and other subjects tested on the bar exam that causes most discipline. Discipline is likely the result of a combination of (1) the personal characteristics that cause a person to prepare inadequately for the bar exam, and (2) the practice settings where those lawyers end up, as well as other factors. But we are not claiming a causal connection between bar exam performance and discipline, as a causal relationship is not necessary to conclude that lowering the bar exam score will increase the rates of discipline.
Moreover, the fact that lawyers with low bar exam scores end up in less sophisticated practice settings, and those practice settings may be the causal factor in higher discipline rates is reason for more concern not less. The practice settings identified are those with the most vulnerable, least sophisticated clients, who are most easily exploited by lawyer misconduct or inattention. Sophisticated corporate clients who can look out for their own interests often hire large law firms where they are not at risk from lowering the bar exam score. It is the least sophisticated clients who need a trustworthy lawyer the most who will be victimized.
I would like to think that our law dean critics could be persuaded that our analysis is offering important, previously unavailable information to an imminent public policy decision. But I will settle for other, more disinterested observers finding our analysis valuable. And most of all, I hope that the media and the public will keep the pressure on the State Bar to investigate the risk of increased discipline, as the State Bar alone controls the most relevant data.
Comments