A Last Word on the Seto Rankings

This is the third (and I hope final) post on Theodore Seto’s article entitled, “Where Do Partners Come From?” published last month in the Journal of Legal Education. Professor Seto’s article ranked law schools by the raw number of partners they have produced in NLJ 100 law firms. In a prior post, I argued that the article was misleading as a guide for students choosing a law school because (among other things) it failed to control for law school class size. The failure to control for class size provided a rankings boost for larger schools simply for being larger, distorting the results from the top school to the bottom school. In this post, I compute size-corrected figures for the data in his article.

To do this, I simply divided the number of partners in Seto’s article by a measure designed to estimate the total number of graduates at each school over the 18 years from 1986 to 2003 (the approximate period during which Seto’s partners would have graduated). To construct this measure, I collected data on total enrollment sizes for full-time and part-time students from the Official Guide to ABA-Approved Law Schools.  I created the Average Class Size measure by taking total full time enrollment divided by three (to account for the three-year full-time JD program) added to part-time enrollment divided by four (to account for a four-year part-time program). 

I collected this data as of two historical years, 1985-1986 (considered one year under the older labeling) and 1999. I chose these two years because the partners in Seto’s study would have graduated mostly between 1986 and 2003 so these measures provide coverage near the beginning and the end of that range. Accordingly, I computed the Average Class Size based on enrollment numbers from the Official Guide for 1985-1986 (near the beginning of the range) and for 1999 (near the end of the range given a seven-year partnership track and the data’s termination point of 2010). I averaged the enrollment figures from these two dates to create a blended Average Class Size over the whole period. I then multiplied this number by 18 years (the approximate period during which Seto’s partners graduated) and used that as the denominator for “Partners per Student.”

I first present in the plot below the raw Seto results (not controlled for law school size) against the 75% percentile LSAT score in 1999 (the earliest date for which I could find reliable LSAT data for all schools). I have taken the natural log of the Seto results to make them linear with LSAT. I have labeled some of the schools that seem to “outperform” and “underperform” their LSAT scores the most. One will note that the outperformers are generally law schools with large enrollments and the underperformers are generally law schools with small enrollments. This plot shows the bias toward large schools that Seto’s approach has hard-wired into the ranking. Boston University is an interesting case because although it is not a huge law school now, it was very large throughout much of the relevant period.

Next I present the same chart with logged Partners per Student (controlled for size) on the y-axis, rather than the raw number of partners. This plot shows a much clearer linear relationship between LSAT and partnership outcomes. The reason is that the irrelevant noise variable of law school size has been controlled for, exposing the underlying relationship more clearly.

To see some of the detail underlying these plots, the following table sets out, in rank order, the results that would have been obtained if Seto had controlled for class size using this method. As I discussed in my prior post, one of the most egregious results from the Seto article was the ranking of Loyola Law School Los Angeles at #25, ahead of the University of Southern California and many other law schools with national reputations. When one controls for class size, Loyola Law School falls from #25 in Seto’s ranking to #50, near where it is ranked in US News.  Indeed, the University of Southern California has twice the rate of big firm partnership on a per-capita basis that Loyola Law School has. It turns out that the “Loyola” that deserves to be ranked near the top 25 is Loyola Chicago, which ranks #26 on a per-capita basis (there are three Loyolas).

The most significant differences between Seto’s results and the size-controlled results were Suffolk, which Seto ranked 40 places higher than it ranks on a per-capita basis, and Washington and Lee, which Seto ranked 39 places lower than it ranks on a per-capita basis. Washington and Lee was ranked 67 in Seto’s article, but on a per-capita basis ranks 28, much closer to its US News rank of 24. Other schools that Seto’s ranking placed at least 25 ranks too high compared to per-capita numbers were Miami, Brooklyn, New York Law School, McGeorge and Loyola Los Angeles. Other schools that Seto’s rankings placed at least 25 ranks too low compared to per-capita numbers were Missouri-Columbia, Arkansas-Fayetteville, Arkansas-Little Rock, and Willamette. In almost all of these cases, the per-capita rankings put each school closer to its US News ranking than did Seto’s results.

I present the following table not as an alternative ranking that prospective applicants should rely on, but as a disclosure that should “bespeak caution” in relying on Seto’s rankings. There are a number of reasons to be cautious in using Seto's rankings besides failure to control for size: (1) I cannot verify whether Seto’s partner counts are correct because I have not been allowed to examine the underlying data, (2) the data include employment outcomes that are hopelessly out of date, as discussed here, (3) Seto collected data on the NLJ 250 firms but mysteriously included only results for the NLJ 100, leading one to wonder how the results would be different if all the data were included, (4) the choice to omit the next 150 firms excludes many top-tier firms with fewer lawyers, such as Wachtell, Lipton (!). Thus, the results are dependent on many things other than law school size, which is all I have corrected here. I hope that readers will comment with any problems they observe in this table, and I will do my best to revise the post to make it as accurate as possible.

Disclaimer: my own institution ranked almost exactly the same in both versions, as it is almost exactly the median size.

New Rank	School name	Seto Rank	Average Class Size	Partners per Student
1	University of Chicago	8	180	0.131481481
2	Northwestern University	10	204.8333	0.098996474
3	Harvard University	1	573.3333	0.091666667
4	Columbia University	5	342	0.083820663
5	University of Virginia	4	378.1667	0.077420303
6	University of Pennsylvania	11	236.4167	0.077311714
7	New York University	3	400.5	0.075322514
8	Stanford University	17	178.3333	0.074766355
9	Yale University	16	199.6667	0.070395103
10	University of Michigan--Ann Arbor	7	363.3333	0.067889908
11	Georgetown University	2	632.75	0.064006322
12	Cornell University	21	179.3333	0.063197026
13	Duke University	19	193.8333	0.062768702
14	Vanderbilt University	22	178.6667	0.057835821
15	University of California--Berkeley	14	281.5	0.05664101
16	George Washington University	6	448.2917	0.055395483
17	University of Notre Dame	28	172	0.051356589
18	University of Illinois	23	204.3333	0.049755302
19	Boston University	12	388.5	0.045331045
20	University of California--Los Angeles	15	325.6667	0.043841692
21	Boston College	20	274.6667	0.043082524
22	University of Texas--Austin	9	502.5	0.042454395
23	University of Southern California	32	200.6667	0.041805094
24	Fordham University	13	407.625	0.041704998
25	Washington University in St. Louis	33	203	0.04077723
26	Loyola University Chicago	29	224.375	0.03813061
27	Emory University	34	237.2917	0.034650278
28	Washington and Lee University	67	121.1667	0.034387895
29	Villanova University	35	223	0.034130543
30	College of William and Mary	48	171.5	0.033689666
31	University of Pittsburgh	39	234.4583	0.030803862
32	Southern Methodist University	38	245.5833	0.029860875
33	University of California-Hastings	18	438.5	0.029519828
34	American University (Washington)	24	340.125	0.029237617
35	University of Minnesota	39	249.5	0.028946782
36	Catholic University of America	36	271.3333	0.027641278
37	University of Kansas	55	175.5	0.027540361
38	University of Missouri	64	158.8333	0.026582721
39	University of North Carolina	46	234.1667	0.025860024
40	University of Houston	31	331.1667	0.025666834
41	University of Georgia	49	217.5	0.025287356
42	Ohio State University (Moritz)	50	217.6667	0.024502297
43	University of Florida (Levin)	29	351.1667	0.024363234
44	Tulane University	41	285.4583	0.024327349
45	University of Wisconsin--Madison	42	280.0833	0.024000793
46	Temple University (Beasley)	26	371.25	0.023943135
47	Indiana University--Bloomington	57	198.9583	0.023734729
48	University of Iowa	53	216.5	0.023351296
49	University of Maryland (Carey)	47	255.9167	0.022793878
50	Loyola Marymount University	25	395.75	0.02274163
51	George Mason University	64	189.5417	0.022275958
52	DePaul University	37	339.6667	0.021916912
53	Case Western Reserve University	56	224.2083	0.021309546
54	Wake Forest University	73	161.9167	0.021272946
55	University of Miami	26	425.5833	0.020886365
56	University of California--Davis	75	164.1667	0.020642978
57	Rutgers --Camden	60	216.8333	0.020497054
58	University of Washington	75	169.125	0.020037776
59	St. John's University	42	338.2917	0.019871084
60	University of San Francisco	64	213.7917	0.019749237
61	Baylor University	84	139.1667	0.019560878
62	Syracuse University	58	232.4167	0.019361778
63	Rutgers--Newark	60	231.375	0.019208836
64	Hofstra University (Deane)	54	258.4167	0.019133613
65	IIT (Chicago-Kent)	45	324.2917	0.019015804
66	Pennsylvania State University	78	170.6667	0.018554688
67	University of Richmond (Williams)	81	152.9583	0.018523563
68	Santa Clara University	52	285.1667	0.01811806
69	University of Missouri--Kansas City	83	155	0.017921147
70	St. Louis University	67	232.5833	0.017914726
71	University of San Diego	51	298.2083	0.017511993
72	Pepperdine University	71	214.3333	0.017107309
73	Brooklyn Law School	44	403.2917	0.016392878
74	Brigham Young University (Clark)	93	148.1667	0.015373078
75	Georgia State University	85	180.5833	0.014766959
76	University of Colorado--Boulder	92	165.0833	0.014470806
77	Albany Law School	75	236.5833	0.014324293
78	Yeshiva University (Cardozo)	63	299	0.014306949
79	Arizona State University (O'Connor)	98	149.3333	0.013764881
80	Florida State University	88	206.5833	0.012639505
81	University of Tennessee--Knoxville	94	178.5	0.012449424
82	Indiana University--Indianapolis	78	254.4167	0.012446774
83	Northeastern University	96	170	0.012418301
84	University of Utah (Quinney)	105	125.5	0.012394865
85	University of Alabama	95	175.1667	0.012369172
86	University of Cincinnati	108	126.2917	0.011877268
87	Pace University	81	245.6667	0.011533243
88	New York Law School	62	387.0833	0.011194833
89	John Marshall Law School	69	377.2083	0.010898781
90	University of South Carolina	85	251.3333	0.01061008
91	University of Arizona (Rogers)	105	148.3333	0.010486891
92	Southwestern Law School	80	312.4583	0.009956883
93	Seton Hall University	72	370	0.00975976
94	Stetson University	100	196.2917	0.009622868
95	SUNY Buffalo Law School	90	255	0.009586057
96	Mercer University (George)	115	149.375	0.009298001
97	University of Connecticut	101	201.2083	0.009111617
98	Suffolk University	58	498.25	0.009031611
99	Vermont Law School	118	143.1667	0.008925107
100	University of Kentucky	118	143.5833	0.008899207
101	University of Mississippi	110	163	0.008861622
102	Washburn University	110	169.1667	0.008538588
103	Texas Tech University	103	199.1667	0.008368201
104	University of Oregon	115	166.3333	0.008350033
105	Creighton University	118	156.2917	0.008175598
106	University of Denver (Sturm)	89	313.75	0.008145197
107	University of Nebraska--Lincoln	125	137.4583	0.008083258
108	Marquette University	123	154.875	0.007532957
109	Drake University	127	142.9583	0.007383659
110	Lewis & Clark College	108	205.0833	0.0073141
111	University of Oklahoma	110	197.6667	0.007307476
112	Louisiana State University	105	216.6667	0.007179487
113	University of Baltimore	96	297.7917	0.007089222
114	Seattle University	99	284.125	0.006843623
115	University of the Pacific-McGeorge	90	381.4167	0.006408856
116	William Mitchell College of Law	103	299.375	0.005567154
117	West Virginia University	141	131.125	0.005507891
118	Gonzaga University	132	174.1667	0.005422648
119	Univ. of Arkansas--Little Rock	148	117.7083	0.00519174
120	Quinnipiac University	125	234.3333	0.004741584
121	Univ. of Arkansas--Fayetteville	148	131	0.00466497
122	Cleveland State (Cleveland-Marshall)	118	275.9167	0.004631028
123	Willamette University (Collins)	148	133.9583	0.004561949
124	Loyola University New Orleans	132	211.5417	0.004464579
125	Samford University (Cumberland)	131	224.6667	0.004451039
126	University of Akron	138	179.9167	0.004322989
127	University of Tulsa	141	182.7917	0.003951068
128	University of Toledo	141	201.5417	0.003583488
129	Wayne State University	148	258.4583	0.002364447