The Most Difficult Bar Exams

Everybody loves rankings, and everybody loves the bar exam. So I thought the combination of the two would make for a winning post as the academic year winds down.

How much do a law graduate's chances to pass the bar depend on which state's examination he or she takes? It turns out that the choice of state makes a big difference. Some states have bars you could trip over if you're not careful, and some are challenging even for the most elite test-takers.

Law students and lawyers evaluate the relative difficulty of bar exams all the time. The most intuitive measure involves simply comparing bar passage rates among the states. That gives some indication of the bar exam's difficulty, but of course the composition of bar takers may vary significantly across states, making such a comparison potentially misleading.

To develop a slightly more accurate assessment of relative bar difficulty, I ran a regression of the bar passage rate for each ABA-accredited school for 2010 and 2011 on the school's median undergraduate GPA and LSAT, with an indicator variable for each state, weighted by the number of takers at each school. I collected this information from the LSAC Official Guide to ABA-Approved Law Schools (Alaska, Delaware, and the DC bars were omitted because of too few observations).

The table below sets out the resulting ranking of state bar exams in order from most difficult to least difficult. For each state, the weighted average LSAT score and the calculated (in the data) and overall (reported by the state) bar passage rates for 2010-2011 is set forth, along with the "bonus percent" relative to the California bar. The "bonus percent" is, roughly speaking, the percentage point bonus for a school's bar passage rate relative to the California bar (with predictors held at their averages).

The conventional wisdom (around here) turns out to be true--the California bar exam is probably the most difficult. Readers of this blog know that I am not exactly a fan of California's state government or its economic climate, but to the extent one thinks a difficult bar exam is desirable, California has accomplished that goal.

The other bar exams toward the top of the list are surprises, at least to me. Virginia turns out to have the bar exam takers with the highest LSATs, and still has a (relatively) low bar passage rate, making it the sixth most difficult bar. Arkansas, Washington, Louisiana, and Nevada are able to rank #2-5 because of very low passage rates with average predictors.

The table also shows how bar passage rates alone would be misleading. One would conclude from overall bar passage rates that the California and Wyoming bar exams had approximately the same difficulty, when in fact they are on opposite ends of the difficulty spectrum. Also, based on bar passage rates, one would conclude that the Michigan and New York bar exams were of almost identical difficulty, when in fact New York is much more difficult. Actually, the Michigan bar exam was not only very easy in 2010-2011, but also had results that didn't correlate much with candidate quality, which is suspicious. Perhaps that is why it was changed dramatically in 2012. More about that in a follow-up post.

What are your chances of passing the bar in your state? Check out my experimental bar examination calculator!

UPDATE 1: Important update on the 2012 Michigan bar exam here.

UPDATE 2: A new ranking with more data can be found here.

UPDATE 3: The ABA is considering a uniform bar passage rate requirement for law schools across all states. Does the chart below look uniform across all states to you?

 

Rank	State Bar Examination	Bonus Percent	Calculated Average LSAT	Calculated Passage Rate	Overall Passage Rate
1	California	0.00	160.92	75.69	72.22
2	Arkansas	1.43	155.21	72.75	73.35
3	Washington	3.29	158.45	76.52	72.82
4	Louisiana	3.78	154.86	73.09	70.00
5	Nevada	4.72	158.41	77.85	72.86
6	Virginia	5.10	163.10	83.43	77.61
7	Oregon	5.19	159.21	78.67	76.04
8	West Virginia	6.47	153.43	76.78	78.80
9	Vermont	7.87	155.04	77.78	81.18
10	Maryland	7.99	160.40	82.82	79.08
11	Florida	9.61	154.54	79.34	77.72
12	Rhode Island	9.77	151.00	77.38	78.44
13	New York	9.78	163.02	87.51	85.72
14	Indiana	10.02	156.85	83.39	81.84
15	Hawaii	10.37	155.97	82.19	81.49
16	Arizona	10.74	158.94	84.97	80.25
17	New Jersey	10.83	158.94	84.24	82.99
18	Kentucky	11.24	156.61	83.68	83.98
19	Utah	11.73	160.95	89.12	89.15
20	Mississippi	12.01	152.58	81.38	82.75
21	North Carolina	12.26	155.84	83.00	77.15
22	Maine	12.26	155.53	83.59	85.43
23	South Carolina	12.61	155.87	82.44	78.94
24	Colorado	13.34	160.01	88.89	84.23
25	Texas	13.77	157.81	86.49	84.08
26	Kansas	14.47	156.18	85.41	88.12
27	Georgia	14.80	159.75	89.25	84.95
28	North Dakota	15.19	151.00	83.64	85.26
29	Idaho	15.34	154.18	84.50	84.25
30	Ohio	15.53	155.82	87.12	86.09
31	Tennessee	15.71	157.82	89.36	83.34
32	Illinois	15.73	159.58	90.44	90.14
33	Connecticut	15.82	157.68	88.06	85.16
34	New Hampshire	16.85	153.57	85.72	84.47
35	Pennsylvania	17.39	157.09	89.28	84.38
36	Minnesota	17.87	157.55	91.20	90.90
37	Massachusetts	18.03	158.39	91.08	89.72
38	Alabama	20.09	157.56	93.50	85.71
39	Missouri	20.32	157.56	93.35	91.03
40	New Mexico	20.34	156.45	91.91	89.47
41	Oklahoma	20.43	154.96	90.45	87.89
42	Michigan	20.57	152.00	87.88	85.85
43	Wyoming	20.83	153.68	91.55	77.28
44	Montana	20.92	155.00	92.59	91.05
45	Iowa	21.34	156.56	93.84	92.13
46	Nebraska	21.44	155.39	93.69	88.88
47	Wisconsin	24.94	159.12	99.73	91.87
48	South Dakota	29.73	150.00	97.22	94.26

RELATED STORIES:

The ABA is considering a uniform bar passage requirement for law schools in Bar Passage and ABA Accreditation.

CAVEAT: The relative rankings of the states are approximate and based on aggregate data that have inherent limitations. This should be obvious, but you should not plan your life around this blog post!

Comments

[Henrique]

It is interesting that New York is perceived as a very difficult exam but ends up in 12th on this list. I think the difference between the perceived difficulty level for NY and its ranking under the methodology is a result of the considerable number of foreign-educated lawyers sitting for the exam in NY.

http://www.nybarexam.org/ExamStats/2012_NYBarExamStatistics.pdf

Because the methodology is based on the relationship between passage rates and LSAT, it does not pick up foreign attorneys (for which the passage rate is lower than JDs from ABA-approved law schools). I wonder if the methodology is using the "ABA passage rate" of the general numbers.

[Delaware Lawyer]

Had Delaware been included in the rankings, it would have probably been up there in the top 10. It is a very hard examination with a very low pass rate (62.7% overall, 69.4% for first-time takers)

http://courts.delaware.gov/bbe/

[ArkansasLawStudent]

I see that you say "weighted LSAT" and it looks like you are weighing that with the average GPA. Some schools might have a higher GPA average but the schools the students are coming from grade easier, hence an inflated difficulty score.
Could you post a regression of the results with an unweighted LSAT so we could see the comparison?

[Aaron]

I have always said that it's unknown how hard the Cali bar actually is, because they allow graduates of non-ABA accredited law schools to sit for it. A more accurate measure of its relative difficulty as compared to other states which require ABA approved law school graduates would only consider those test-takers' percentages.

____

J. D. degree from a law school accredited by the State Bar of California or approved by the ABA;
Four years of study at a fixed-facility law school registered with the Committee;
Four years of study, with a minimum of 864 hours of preparation and study per year, at an unaccredited distance-learning or correspondence law school registered with the Committee;
Four years of study in the law office/judge’s chambers study program; or
A combination of these methods.

[Robert Anderson]

Aaron,

Thanks for the comment. The analysis only includes schools accredited by the ABA as it is taken from the LSAC Official Guide to ABA-Approved Law Schools. I do not incorporate the overall pass rate for all takers in any way, so the analysis does exactly what you suggest.

Regards,
RA

[Athina]

New York exam has nothing to do with the CA bar exam. Trust me.!!
I passed both bars.I found the CA bar very hard and demanding real and not superficial knowledge of the law .

[LglBgl]

Well, omitting DE doesn't do you any favors. Everyone knows they have the hardest bar.

[David Hiersekorn]

I don't agree with the methodology. They accept California as they hardest exam, which is a fair assumption. Then, they compare each school's pass rates in other states to that school's pass rates in California.

For example, if Harvard grads pass 76% of the time in California and 81% of the time in Arkansas, then Arkansas is 5% "easier." If you take that method and apply it across all schools, you get a "bonus" factor. They used that bonus factor to determine the relative difficulty of each state's exam.

However, that method is really measuring each state's ability to attract the best students from top schools. The better a state does at attracting top talent, the "easier" they will rank on this methodology. To illustrate this, suppose that Harvard graduates 100 people. Suppose that the top 50% of Harvard's graduates take the New York exam and the bottom 50% take the Arkansas exam. The top 50 would do well in New York, even in spite of it's difficulty. After all, the top 50% of Harvard grads are likely to pass ANY exam. On the other hand, if there are any "failers" in the Harvard batch, they will be in the bottom 50%. So, those failers take the Arkansas exam and fail.

Poof! Arkansas is harder than New York. Adding California as a benchmark doesn't change the inherent flaw in the method. It only presents a more complex version of the same flaw. It makes this a measure of the likelihood of any given state to attract candidates from a school who are better/worse than the candidates that same school sends to California.

That methodology won't stand. A better approach would be to use raw MBE scores and compare pass rates against MBE scores. You could, for example, compare the average MBE score among passers. I believe that would be a better and more accurate measure.

[Nope]

The Kansas bar is not nearly as difficult as the Missouri bar. Ask anyone in the LC area who has taken both.

[Evan]

You should take into account the UBE states (11 I believe) which all give the exact same test (as far as I know)...

[Jeff]

As somebody who took the bar in Texas, I got the impression(from sitting in the room) that a lot of lawyers from out-of-state schools took the bar there. They seemed to struggle with a lot of Texas's intricate laws or the fact you need to know O&G for the bar.

Point I'm making- can you include in this data the % of test-takers from in-state vs. out of state. It would seem that if everyone from Texas goes to Arkansas and aces that bar, that would be a good gauge of how 'difficult' that exam was. Your model just shows that Arkansas students who take the bar might just not be cut to be lawyers.

I know the state of Texas releases information about in-state vs. out-of-state educated bar takers. Usually the in-state is 10-15% higher.

[Robert Anderson]

David,

Thanks for the comment. It is true that the methodology assumes that all of the takers from each school are drawn from the same distribution in every state. So if it is true that the lower-ranked students tend to take the bar in different states than the higher ranked students, then the states where the higher ranked students go will look easier. However, when you look at the data you will see that in almost all states the majority of takers are from in-state schools, and the majority of the takers in most schools take the bar in state. So as a practical matter I think the impact of the possible pattern you identify is not likely to affect the results much, but I cannot exclude it.

Unfortunately, I don't have data on individual test-takers or their MBE scores, so I can't perform the analysis you suggest. I wish I could!

[Robert Anderson]

Evan,

This is a good point, except that other than the MBE which the vast majority of states use and is graded nationwide, the states grade the remainder of exams themselves. The very same essay questions can be "difficult" or "easy" depending on the criterion for passage applied.

Best,
RA

[DE gal]

I've personally sat for the Pennsylvania, New Jersey, and Delaware bar exams and I must say that the Delaware bar exam was the most brutal of them all. Low pass rate, monopolized and unorganized bar prep class, a ridiculous amount of material including most sections of the UCC makes me believe this exam would be up near the top of the list.

[CA Taker]

I agree with the previous post. Delaware has a bar exam that is just as difficult as California's Bar Exam. Like, California, Delaware has a 2 1/1 day exam. It also has 8 or 9 essays and 2 MPT's. The essays are 45 minutes and MPT's are 90 minutes. Delaware probably has low numbers for your methodology because they only offer their exam once a year and it is a small state. However, I believe it was a mistake for you to leave it out because I've taken both exam's and can tell you from personal experience that they are both demanding. I would say Delaware might be a little more difficult on the substantive side because there are a lot of unique caveats to Delaware state law that is different from common law principles.

[CA Taker]

I agree with the previous post. I bet your thinking that the Delaware Bar Exam is easy because Joe Biden was able to pass it! However, from personal experience I can tell you that the Delaware Bar Exam is just as difficult as the California Bar Exam. It was a mistake for you not to include it on your list. Delaware probably has low numbers for your methodology because Delaware is a small state and only offers their bar exam once a year. Delaware also has only one law school. However, that shouldn't impact the fact that its just as difficult of an exam as California's. Like California, the Delaware Bar is a 2 1/2 day exam. It also has 8 or 9 essays, which is 2 or 3 more essays than California requires, and also has 2 MPT's. There is 45 minutes for each essay and 90 for each MPT. If I had to choose between the two, from personal experience, I would have to say Delaware might be more difficult on the substantive side. Their are a lot of caveats to Delaware State Law that isn't taught in most law schools. However, California's is also difficult because you really have know how to analyze facts and be good at legal writing. I would defiantly say California's MPT's are harder. However, Delaware requires you to write just as an in depth answer as California does on their essays. I found Delaware to be very difficult because you had less time to do that in. Its very easy for a smart person to fail the Delaware Bar because they ran out of time. Just my two cents worth!

[Robert Anderson]

Dear CA Taker,

Regarding Delaware, I would have liked to include it, but there was literally no data. Even the Delaware school (Widener) reported only Pennsylvania bar exam results. So unlike Alaska, which actually had some data but numbers I thought were too low, there was actually nothing to say at all about Delaware. So it may well be a very difficult exam, but I am unable to say with the data I have available.

Best,
RA

[CA Taker]

I think the Weidner Law library might have the numbers you are looking for. I know they keep records of old exams and might have statistical information. If not, the Weidner Bar review course might have them as well. Its a separate course from BARBRI but I am pretty sure they keep records of who passes and doesn't as well as LSAT scores, etc. If not, you can always call the State Bar office. Also, I accidentally submitted the post twice. I didn't think it was accepted the first time. Sorry about that.

[aa]

What about DC?

[bjstalder]

I totally disagree with this chart. California, Texas and New York are the most difficult.

[Robert Anderson]

Dear bjstalder,

I don't think it's accurate to say that you "totally disagree" with the chart, considering that California is the most difficult on the chart and New York is also ranked as quite difficult.

But with all due respect, is your assessment based on anything in particular other than that these are the three most populous states?

Best,
RA

[BrianL]

I still maintain the position that a grad from an ABA law school, if the ABA was relevant, should not be required to take an exam.

The States should offer an exam tomorrow grads of non-ABA schools and all state should recognize credentials from other States.

When I took the Florida bar, if the taker got a high enough multistate score he could be admitted automatically to the Virginia and Colorado bars and to the D.C. bar through Virginia. I qualified. This cross licensing should be expanded and promoted to all States.

I am not dicing the licensing of attorneys in a biased manner as I am also a retired CPA and have similar issues with the CPA licensing methodology, in fact moreso.

Brian Levy, JD
Sent from my Playbook

[bb]

I've taken a few bar exams including Delaware. Delaware is BY FAR one of the hardest exams in the nation. The overall passage rate is rarely over 60%. In addition to what has already been mentioned, examinees must have a thorough knowledge of Delaware corporate law, partnerships/agency and Chancery Court procedure. Each essay is graded on a curve with a mean of 50 which means you can have more points deducted then added to your overall score if you do well on essays that other testtakers do well on.

[Robert Anderson]

Dear bb, thanks for the comment! A number of people have shown me data that would suggest you're right. Unfortunately none of the schools disclosed bar passage rates for Delaware (perhaps because the bar is difficult) and as a result I was compelled to leave it out. One resource that supports the idea that Delaware is the most difficult is Chart 9 of the Comprehensive Guide to Bar Admission Requirements published by the NCBEX, which shows Delaware has the highest required passing score on the multistate bar exam. Best, RA.

[David Shelvey]

Another issue is Washington has an essay only exam. There are no multiple choice questions which means you have to memorize terms of art and apply them in a sentence. You cannot simply say, 1 of the 4 selections is a correct answer. I had a bad LSAT score, but a good GPA which got me into law school. I sucked at M/C questions, but aced a lot of my essay exams. I will add I am currently waiting on my WA BAR results. WA is now going to the UBE M/C this summer with the WA specific testing online.