Citation counts (and other types of citation-based metrics) are increasing in importance in the legal academy. Some people like the objectivity of these measures and others lament their failure to capture important non-quantifiable aspects of scholarly influence.
One of the most influential citation count rankings in the legal academy is the Sisk-Leiter approach that Greg Sisk updates every three years. Last fall when the new Sisk et al. citation count study came out I proposed a small change to the Sisk-Leiter method that would attempt to measure engagement, defined as citing a particular article more than once. This was designed to address the "throwaway" citation problem that critics of citation counts have raised--that some papers may receive a large number of perfunctory "one off" citations that are less meaningful as a measure of scholarly influence.
I decided to experiment with this using the most highly cited scholars in my field, corporate law and securities regulation, using the citation count rankings as compiled by Brian Leiter. I used the same method I proposed in my previous blog post (with some slight technical changes I'm happy to discuss for those interested). For the twenty scholars listed on that ranking, below is the relationship between engagement counts and citation counts.
Although the two measures do produce slightly different ordinal rankings, the differences are not significant at least for this small group of 20 scholars. Professor Stephen Bainbridge moves up from third place to second place and Professor Donald Langevoort moves up from 7th place to 5th place. Although these differences are not large, it is possible there are large differences for some types of scholars that simply aren't represented here. Still, a blend of the two measures might produce a better overall ranking for those cases of divergence, even if the overall differences are relatively modest.
Comments
You can follow this conversation by subscribing to the comment feed for this post.