More on the Most Difficult Bar Exams

In last week's post I gathered data to produce a rough comparison of bar examination difficulty across the various states. In response to (numerous) comments, I created this new table that changes the approach in two ways that may make the results more accurate:

• I have gathered more data to include four years of bar examinations (2008-2011) rather than two years. This should allay concerns that the results are attributable to a fluke of the last two years.
• LSAT is controlled in this version but not undergraduate GPA. Some commenters suggested (with reason) that undergraduate GPAs may not be comparable across states in the same way LSAT scores are.

The results are largely the same as in the prior table, with the exception that Louisiana takes the second position in this table. Note that this data is really only reliable for the larger states where that have a significant number of exam takers in the data. The rankings of the larger states hardly changed at all by adding the additional data or the slight change in methodology. Some responses to common comments are below.

Rank	State Bar Examination	BonusPercent	Calculated Average LSAT	Calculated Passage Rate	Overall Passage Rate
1	California	0	160.68	76.85	73.41
2	Louisiana	0.91	154.79	72.07	68.23
3	Washington	3.90	158.12	78.27	74.54
4	Oregon	5.05	158.83	80.11	77.96
5	Nevada	5.19	158.09	79.53	74.28
6	Virginia	5.56	162.82	84.47	78.73
7	Arkansas	5.60	154.45	76.43	77.35
8	West Virginia	5.72	153.22	75.36	77.56
9	Maryland	7.02	159.85	83.06	80.08
10	Vermont	8.44	155.02	79.82	82.24
11	New Jersey	8.98	158.65	83.86	83.12
12	New York	9.14	162.89	88.13	86.49
13	Rhode Island	9.28	151.53	77.29	78.90
14	Colorado	9.88	159.64	85.72	82.58
15	Florida	10.02	154.40	80.80	78.44
16	South Carolina	10.29	156.01	82.62	80.41
17	Arizona	11.27	159.26	86.74	80.83
18	Utah	11.39	160.97	88.52	87.56
19	Hawaii	11.43	155.99	83.74	83.30
20	Indiana	11.46	156.78	84.54	83.01
21	North Carolina	11.88	156.56	84.74	78.18
22	Texas	12.27	157.86	86.38	84.38
23	Idaho	12.56	154.32	83.26	82.31
24	Wyoming	12.93	153.32	82.67	72.83
25	Kentucky	13.57	156.59	86.46	84.61
26	Georgia	14.36	160.39	90.93	85.99
27	Illinois	14.86	159.68	90.74	89.97
28	Connecticut	15.32	157.32	88.92	86.64
29	Pennsylvania	15.49	157.20	88.97	84.61
30	Tennessee	15.82	158.31	90.38	84.83
31	Ohio	16.00	155.54	87.88	86.69
32	Maine	16.06	155.29	87.70	86.58
33	Massachusetts	16.07	158.30	90.62	90.16
34	Kansas	16.40	155.49	88.23	89.22
35	New Hampshire	16.76	153.29	86.47	86.09
36	North Dakota	17.18	151.00	84.67	83.23
37	Mississippi	17.31	152.64	86.39	85.35
38	Minnesota	18.27	157.27	91.82	91.46
39	New Mexico	18.51	155.56	90.41	87.26
40	Michigan	18.67	151.57	86.72	84.98
41	Montana	18.97	154.54	89.89	90.73
42	Iowa	19.81	156.69	92.81	90.96
43	Missouri	20.18	157.30	93.76	91.20
44	Oklahoma	20.73	154.49	91.60	89.85
45	Nebraska	20.79	155.27	92.41	88.90
46	Alabama	21.13	157.33	94.75	87.16
47	Wisconsin	23.51	159.66	99.37	91.83
48	South Dakota	28.00	150.51	95.02	93.08

 CAVEAT: The relative rankings of the states are approximate and based on aggregate data that have inherent limitations. This should be obvious, but you should not plan your life around this blog post!

Some responses to common comments:

• Several people have suggested that Delaware may have a bar exam comparable to that of California in difficulty. Unfortunately the data available do not include results for Delaware, so I an unable to include it in the table.
• These results are subject to the usual caveats about drawing inferences from aggregate data, but unfortunately I have little choice because individual test taker data are not available to me. 
• Yes, I know that Wisconsin school graduates have a "diploma privilege" and that the 100% bar passage rate is attributable to that privilege. I personally think that a bar you pass automatically is the easiest bar of all, but acknowledge that taking out the diploma privilege effect would make Wisconsin look somewhat less ridiculously easy.

Comments

[JGM]

It would be interesting to compare the rates of test-takers who grauated from a law school in the state with those who did not.

[Casp7]

The reason New York appears easier than one would think is that school-wide LSAT data is used in this ranking to determine the quality of a state's test-taker pool. On average, and especially for those coming from law schools outside New York, new attorneys hired in New York will tend to be among the higher performing students at their respective schools, so this ranking methodology will understate the pool quality. The data isn't available to prove it, of course, but I'd be surprised if that skewing effect, in that direction, weren't far more of an issue for New York than for any other state. (That is to say, New York is more of a top-student-magnet, by far, than any other state.)

[Tony]

Where does the Uniform Bar Exam (UBE) rank relative to the individual state exams in degree of difficulty?

[Robert Anderson]

Hi Tony,

We really can't say in the abstract because states still retain their own passing scores. So although the UBE means that the test itself is the same across jurisdictions, it's really the passing score that will determine how difficult the exam is to pass.

Best,
RA

[Charity]

Do you know what the passing score for South Dakota is?

[Steve]

Would you say that a bar exam with a higher passage rate would be easier or would you look only at the passing score? For example, DC only had a passage rate of 46% in Feb. 2013 but the passing score is 133 which is the same as Iowa, Kansas, New Jersey & New York. However, those states had much higher passage rates of 80-90%.

[RWL]

Find a way to take out the graduates of all the unaccredited law schools in California and you might start getting closer to something that is more accurate. I'm confused about Nevada though--is it hard because the professionalism rules are out the window in Nevada???? At least where I'm licensed, a state just as high an ave LSAT as CA, we can't dream of having a law firm called "Half-Priced Lawyers" or say, "you won't just get paid, you'll get EVEN"! What a joke. Truth is, you honestly cannot measure anything accurately because the LSAT is a joke, undergrad GPAs are vastly different, and you have a magic "bonus" percent that is nothing more than a little bit of Al Gore math.

[Robert Anderson]

RWL: Thanks for the comment. However, the analysis does not take into account unaccredited schools in any way, so they do not affect the analysis. Nevada's difficulty is likely the result of their "conjunctive" passing rule, which requires applicants to pass each of two separate parts of the exam. This makes it more difficult to pass the exam. As for whether the LSAT is "a joke," I'm not exactly sure what you mean. The LSAT is a very good predictor of law school grades and bar passage. Perhaps you might argue all of those things are "jokes," but I suppose that depends on one's sense of humor!

[rw970]

I can't believe the New Jersey exam is ranked harder than New York's. Having taken both at the same time (like many of us New York law students), I'm pretty sure of this.

99.99% of my study time was for the New York Bar - I only glanced at the New Jersey Kaplan's "book" the night before. This is because New York tests on their own versions of the 6 MBE courses + like 15 other New York specific ones. New Jersey tests you on the 6 MBE courses (not even Jersey-specific) and New Jersey civil procedure.

If you study enough to pass the MBE, you're virtually guaranteed to pass the NJ bar exam. The only reason I can think of why NJ has a lower passage rate would be that there's basically only two law schools in NJ, Seton Hall and Rutgers.

[Paula]

This chart does not take into account that certain states allow people to take the bar without ever having gone to law school . Example, California. Cali allows people to take the bar exam with 4 years of a Law Office Study Program. In fact, these persons do not even have to have a college degree. According to The California Bar Admissions Committee statistics, these test takers have a 18% passage rate for July 2011. Foreign attorneys have have only a 21% passage rate. These numbers greatly bring overall passage rate down. So this chart is not a fair assessment because most other states do not allow this. Those who graduated law school from an ABA accredited school have a passage rate of approximately 70% for first time takers.

[Paula]

I meant 76% passage rate. Typo above

[Robert Anderson]

Hi Paula,

These figures are based on graduates of ABA-accredited schools only, so the other categories you mention do not affect the rankings.

Best,
RA

[Badel]

I know I'm a bit late for the comments but I would recommend you take a look at the Puerto Rico bar passing rates. Our average for the past 4 years barely makes about 35% passing I believe.

[Ken Gallant]

For some states, such as Arkansas, there is a fairly good way to judge difficulty directly. The grading system here norms all the essay questions to the Multistate multiple choice exam, with a "passing" score of 135 Multistate adjusted score.
So, if 70% of the students receive a score of 135 or better on the multistate, then 70% of the students will "pass" on each essay question as well. The highest grade on each essay will be equivalent to the highest multistate score as well, and so on down. If 80% receive a grade of 135 or better, 80% pass each written question. [So, you really ought to root for everyone to do well on the multistate, but that's beside the point.]

I understand that many, but not all, states use a similar system. For each of those states, you can tell how hard they are simply by knowing what multistate grade the test "norms" to. A state "norming to 135" is harder than a state norming to 130.

So, I suggest you check that out--should not be that hard to discover.