The pass rate on the California bar exam last July was the lowest since 1984. This has sent many people scrambling for an explanation for the dive in the pass rate. In reality, however, the reason for the drop is not mysterious. The main reason the bar pass rate dropped is that as law school applications declined beginning in 2010, schools have admitted weaker and weaker incoming classes that were less able to pass the bar. Although other factors likely played a role, especially the increasingly predatory law school transfer market, the decline in applicant quality is the main cause of the current "crisis" in the California bar exam.
The stress of declining bar pass rates has prompted many critics of the bar exam to emerge, especially among California law school deans. The California deans predictably contend that the passing score for the bar exam is too high in California. Other critics (including some law school deans) go farther, however, arguing that the exam lacks "validity" even if the passing score were changed. That is, the critics believe the exam doesn't test the skills that it is intended to test, regardless of how difficult it is to pass. According to these critics, although lowering the passing score in California might help, really the solution is to revamp the test altogether.
I refer to these critics as "bar exam skeptics," in the sense that they express profound doubt about the conventional wisdom that the bar exam tests "minimal competence" to practice law. One of the more thoughtful skeptics is Professor Deborah Merritt, who has written in many venues but most recently in this piece that the bar exam is not measuring the right type of skills needed for law practice. Professor Merritt argues that "growing evidence suggests that our exam is invalid: the knowledge and skills tested by the exam vary too greatly from the ones clients require from their lawyers." As a result, Professor Merritt concludes that "we must develop a definition of minimum competence that tracks the real work of new lawyers."
One one hand, it is certain that the bar exam does not directly test many of the skills identified by Professor Merritt, such as negotiating and interviewing. It is also certain that these are important skills for most types of law practice.
One the other hand, there is a big problem for those who contend that the bar exam does not test the skills needed for law practice. The score on the bar exam tends to be very highly correlated with law school GPA. The correlation is approximately .68, according to this New York Bar study (see Table 5.15), meaning that law school GPA explains about half the variation in bar exam scores. This figure is consistent with other analysis by the National Conference of Bar Examiners, which reports that law school GPA correlates with the Multistate Bar Exam score at .55-.70. These very high correlations strongly suggest the bar exam and law school grades are measuring the same type of abilities. Indeed, first year law school GPAs typically correlate with upper-division GPAs at about .75, suggesting that law school grades are almost as good at predicting the bar exam score as they are in predicting other law school grades.
Why is this significant? It sounds like this information would suggest the bar exam is largely an echo of law school grades, and to that extent unnecessary. If bar exam scores are not measuring the right things for law practice and law school grades are measuring substantially the same things as bar exam scores, then law school grades are not measuring the right things for law practice either. Isn't it possible that law school grades are just as useless in measuring the abilities necessary for successful practice of law?
The problem with this interpretation is that those who have the most experience with the skills needed for law practice--law firms--rely almost exclusively on law school grades and law school rank in making hiring decisions. Law firms seem to believe that law school grades and law school rank are the most important available measures of what is needed for successful law practice. Given that law school grades and law school rank measure substantially the same thing as the bar exam, it seems unavoidable that the bar exam also measures the qualities law firms demand from new lawyers.
It is true, as Professor Merritt notes, that in surveys and questionnaires law firms often identify qualities other than grades or cognitive skills as the skills necessary for success in law practice. But that does not necessarily mean that those other skills are as important than cognitive ones, or really even close. It is simply that the law firm already has a very accurate measure of the cognitive skills (law school grades and bar passage), and it does not have a ready measure of these other skills. Law firms do not have the experience of unknowingly hiring new attorneys who lack the cognitive abilities for law practice, because they already have an excellent measure of those abilities. If law firms really believed these other skills were as important as law school grades, the interview process for new associates would look very different.
Of course, it is possible that the revealed preferences of law firms are actually irrational, and that they should value these other qualities and skills more than cognitive ones. Indeed, there is evidence that law firms systematically overvalue law school rank in hiring decisions, which is the subject of an analysis I've been working on. But law school GPA is so salient, so visible, so central to the hiring process at law firms that it seems unlikely that law firms are mistaken about its relevance. And if the abilities measured by law school GPA are important practice-oriented skills, then bar exam scores are necessarily important practice-oriented skills, as they are essentially mirrors of each other.
None of this is saying that passing the bar exam is an important piece of information in most cases. First, in most jurisdictions (other than California), the bar exam is sort of the cognitive equivalent of tee-ball, where you pretty much need to whiff not to get a base hit. As a result, passing the bar is not a significant hurdle. Second, law school grades are probably a more precise measure of the same set of skills tested on the bar exam. As a result, the bar exam provides little information that is not already available from a law school transcript. But the bar exam does serve an important function in checking the "social promotion" in law schools that allows the most egregiously unqualified students to graduate and practice law. In that sense, it may protect the public from law schools that would otherwise lower their academic standards so far that admission the the bar would be meaningless.
In summary, to argue that the bar exam does not test the skills needed for law practice is to argue that law school grades to not test the skills needed for law practice. And to argue that law school grades do not test the skills needed for law practice is to argue that law firms are profoundly mistaken about the skills needed for law practice. That is a debate I am willing to entertain, but it's an uphill argument for law school professors and deans to argue that they know more about the skills needed for practicing law than do law firms themselves.
Comments