The U.S. News & World Report recently released this year’s law school rankings. Some schools rose and some fell. But who cares?

Despite the lies that can go into calculating them, it appears the law school rankings are still relevant. At least to some folks. I only knew they were released today because of a post on Facebook. A fellow alum lamented my alma mater’s precipitous drop in the rankings. As it turns out, the gist of his gripe was based on competitiveness. Many of his colleagues graduated from ASU, which climbed above the University of Pittsburgh, my alma mater, in the rankings. But can one magazine’s ranking of legal institutions matter for more than just ribbing other lawyers?

The Fear of Looking Bad

During my time there, Pitt went from #57 to #73 in the rankings. The student population freaked out. It was all anyone talked about. Did we make a poor investment? Should we transfer? Would our school’s reputation hinder our job search?

But really we were just scared. Nobody talked about it outright, but that was the undercurrent at the school. We didn’t want potential employers (or later our colleagues and opponents) to judge us for this rankings drop. Finally, we were scared that the education we received would be worse than students at seventy-two other schools, and as a result maybe we wouldn’t be able to keep up in our (hopefully forthcoming) jobs.

The Flaws in the Ranking System

Even if schools all give totally accurate responses to the magazine’s questions, the system is inherently flawed. Steven Harper at The Chronicle of Higher Education delves into the quality assessment component:

Quality assessment is the biggest contributor to a law school’s U.S. News ranking, accounting for 40 percent of its total score. The category itself is a misnomer because it doesn’t reflect quality at all. Rather, using statistically suspect samples of scholars and practicing lawyers, it’s a superficial and unreliable assessment of a school’s reputation.

Where does the assessment come from, you might ask. Harper goes on to explain:

Twenty-five percent of every law school’s total score comes from responses to a survey that the magazine sends each year to four people at every ABA-accredited law school: the dean, the dean of academic affairs, the chair of faculty appointments, and the most recently tenured faculty member. The four receive a list of the country’s accredited law schools, along with a request to rate each—all 195 in 2012—on a scale of 1 to 5. The survey doesn’t ask those responding if they’ve ever set foot on the campuses, met any of the faculty, or have any familiarity with the schools they’re reviewing. Respondents have a “don’t know” option, but U.S. News doesn’t disclose how many people use it for particular schools on the list. After all, such meaningful information would undermine the reported response rate, which was 66 percent for the rankings published in 2011 and 63 percent for 2012.

The rest of the quality-assessment component—accounting for 15 percent of the total U.S. News score—comes from a similar 1-to-5 survey that goes to unnamed “legal professionals, including the hiring partners of law firms, state attorneys general, and selected federal and state judges.” The response rate is abysmal—14 percent for the 2011 rankings, for example, and 12 percent for 2012. Here again, the extent to which respondents replied “don’t know” isn’t disclosed. And whether someone returning the survey actually knows anything about any law school that he or she rates is irrelevant. As a matter of statistical procedure, the entire sampling process suffers from numerous deficiencies.

The Reality of the Rankings

As law students, there is a lot of focus on the rankings. But from my brief time as a lawyer, I can say that focus completely evaporates upon graduation. I’ve never been asked about the rankings in an interview. I don’t judge other attorneys based on which school they go to. Nor do I expect any unkind remarks due to Pitt’s recent drop.

But the law schools continue to feed into the importance of the rankings. For example, this evening I received an e-mail from the University of Pittsburgh School of Law Alumni Association with a special message from our new dean. He opens the letter with “as you have already heard,” indicating that this should be on everyone’s radar. But if I hadn’t been waiting in the courthouse and messing around on Facebook, I never would have known. The message validates the rankings as a system the law school should compete in.

These rankings, from my observation, do not give us any kind of objective basis to judge schools. They don’t help people get jobs, or keep people from getting jobs. Yet law students and academics continue to focus on them, seemingly for the very reasons they aren’t important.

(photo: http://www.flickr.com/photos/evelynishere/3417340248/)