Data is not the plural of anecdote: Legal Scholarship Edition

After a long hiatus due to the weekend and Independence Day, I finally found something that piqued my interest to blog about: a debate about the usefulness of legal scholarship in regards to judging and legal practice. Chief Justice Roberts, in the Fourth Circuit Judicial Conference, noted that law review articles are often useless to judges because they focus on arcane philosophical points rather than resolving difficulties in doctrine or exploring important areas of law. In response, law professor Sherrilyn Ifill penned a response, as posted by Danielle Citron at Concurring Opinions, arguing that law review articles do in fact do doctrinal lifting if only judges would read them. For her argument, she points to a 2007 article used in a recent Court of Appeals case regarding 4th Amendment searches and GPS technology. She then goes on to list 4 examples of law review articles dealing with recent doctrinal issues, which she rounds out with the statement:

“Hutchins’ article is hardly an anomaly.  A recent review of articles posted on the Social Services Research Network, revealed a treasure trove of excellent articles that would greatly assist judges in their work.”

Despite these examples however, Professor Ifill fails to make the argument that she thought she has in the conclusion that law review articles are generally useful to judges if they would read them. She makes a total of three mistakes in her argument:

Mistake One: A few data points do not make a general trend. Her argument supposes that these articles are representative of the general population of articles. However, 5 examples are nowhere near the number necessary to even begin making descriptions using statistical methods. In order to get the gravity of the argument, we must first know the entire population of articles. Taking information from the Washington and Lee Law Journal rankings,  they contain 986 U.S.-based English law journals. Already, our five examples are lost adrift a sea of journals as they constitute less than one percent. Some may contend that some journals are better quality than others, but such arguments necessarily delve into what “quality” means, and whether it is related to the concept of “useful to judges.” Taking our numbers a step further, we need to know how many articles each journal publishes. Using George Mason Law Review as a measuring stick, let’s assume that a law review publishes four articles per issue (not including notes as I want to look at professorial output, not student output) and four issues per year (in the end, some journals publish more per issue but less issues, etc. so it balances out). This leaves us with a total of 15,776 articles per year (4*4*986). So, five articles out of 15,776 comes out to 0.00316, definitely not representative of the population of law review articles. In this instance, what little support Professor Ifill presents is definitely insufficient to make her case.

Mistake Two: SSRN != Published != SSRN. (For those who are not familiar, != is “does not equal in Java.”) Four of the examples provided by Professor Ifill are from SSRN, which is a self-listing by professors of their work, including everything from just abstracts to published works. Although I am a huge fan of SSRN (link to my author page is in the About page), SSRN is not sufficient to reach any conclusions about the state of legal scholarship. First, many of the articles listed on SSRN are works in progress or attempts by authors to draw attention to recent scholarship which has yet to be chosen for publication. Second, SSRN does not include every work published in a law review, so in some ways it would tend to undercount some scholarship while overcounting others. Some academics are not on SSRN, while others update their pages sparingly, and some may not list publications on their directly to draw attention to their major works. The double disconnect means that we have trouble drawing the analogy about legal academia from SSRN as it fails to work in either direction.

Mistake Three: Academia as a category is nearly nonsensical. By attempting to argue that the majority of scholarship is useful, Professor Ifill glosses over the individual characteristics and motivations of particular research. Some pieces are not even intended for judicial consumption, and are targeted toward legislators and other policy makers considering a certain course of action. Although I am unfamiliar with the pieces she references, I do have personal experience with a piece recently cited by the 7th Circuit Court of Appeals that was intended for judicial consumption to resolve doctrinal issues. Before I became the analyst for FantasySCOTUS, Josh gave an assignment to all of his potential interns to work on a segment of the Pandora’s Box. From this perspective, I knew that the main reason behind the article was to discuss the impact of potential victory for McDonald v. Chicago, and to chart the course of law after that decision. Like Pandora’s Box, I’m certain that many of the articles referenced by Professor Ifill are the conscious decision of authors to solve a particularly thorny problem, and could be the product of a practically minded author (both Josh and his coauthor Ilya Shapiro are not tenure track professors). The idiosyncratic nature of both articles and authors would tend to increase the threshold amount of data necessary to make the argument that legal academia in general is useful to justices in doctrinal issues.

In conclusion, Professor Ifill failed to make a persuasive case that legal academia is useful. However, I would also argue that not all legal scholarship SHOULD be useful in that sense. We need high ideas, programmatic comparisons, and doctrinal guidance in order to make any system work, and legal academia is no different. Maybe part of my interest is that my first article is mainly an ideas piece about a novel approach to Supreme Court predictions. However, I feel strongly that not enough scholars make the effort to connect their idea pieces to doctrine or legal practice, and that is what judges are picking up on. For example, my first piece is only the beginning of many, and as I move through my career, I am looking for plenty of opportunities to apply the ideas to different situations such as litigation and cert grants. However, an alternative method of making that connect is to engage in blogging and other forms of communication. By taking the time to show how “the effect of Kant on the evidentiary rules of Bulgaria” relates to something occurring in our legal system right now, and might provide an answer to a question plaguing courts and litigants, we advance scholarship into a happy medium of ideas and action.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: