Sallyportal: Madly Blogging Reed

Guesswork and the Rankings Game

By Chris Lydgate ’90 on September 11, 2014 10:50 AM

Meaningless arrows, dubious data.

U.S. News & World Report released the 30th annual edition of its influential “best colleges” rankings yesterday. You’ll find the usual suspects at the top of the list. Reed came in at the weirdly low Number 77.

To me, however, the biggest surprise was St. John’s College.

St. John’s, if you’re not familiar with it, is a wonderful college in Annapolis, Maryland. Founded in 1696, it boasts an incredible 8-1 student-faculty ratio and a “Great Books” curriculum that makes me drool.

This year, it catapulted from Number 123 to Number 56 in the U.S. News rankings. Remarkably, St. John’s did not achieve this feat by hiring a raft of new professors or recruiting busloads of freshmen with killer SATs.

It just sent back a survey.

Back in the mid-nineties, Reed and a handful of other liberal arts colleges, including St. John’s, pulled out of the US News rankings because of concerns that the enterprise was deeply flawed and vulnerable to manipulation. US News continued to rank the holdouts anyway—possibly fearing that if it didn’t, more schools would bolt.

For almost 20 years, Reed and the holdouts have been relegated to the middle of the pack, although U.S. News has always denied any element of retaliation.

But St. John’s meteoric rise has to make you wonder. Was the school’s score all these years artificially depressed by its refusal to play the rankings game?

It’s worth noting, by the way, that St. John’s president, Christopher B. Nelson, remains a prominent critic of the U.S. News system, despite his school’s sudden reversal of fortune. “It should suggest something about the unreliability of the rankings, frankly,” Nelson told the Washington Post.

I doubt that St, John’s has changed very much over the last couple of years. My hunch is that U.S. News was using estimates, not data, to calculate St. John’s score—and that the estimates were weirdly low.

Funny, that.

Tags: