Alec Ward | The numbers racket

Talking Backward | Why I'm pleased at Penn's rankings slide

By Alec Ward 09/22/15 11:06pm

I suppose I should begin with a disclaimer that I don’t know for certain exactly why Penn dropped from 8th to 9th in this year’s U.S. News and World Report’s “Best Colleges” rankings. What I do know is that I won’t be losing any sleep over it, save perhaps the time it will take me to write this column. As a matter of fact, I’m actually somewhat pleased about the rankings slide.

Don’t get me wrong — I like external validation (almost) as much as the next Quaker. U.S. News’ approval, however, is something I can do without, given what’s necessary to gain it.

To justify this, it’s important to understand the rankings methodology in some detail. The system used to rate and rank colleges is pretty complex, even by U.S. News’ own account. What it boils down to as far as Penn is concerned, however, is this: Competition at the top is fierce. Penn is realistically competing for the top spots with only a handful of prestigious institutions. These schools all tend to score comparably well on the broad categories like reputation, retention and faculty resources, which make up a majority of the available points.

By their very nature, all of these schools graduate a high number of successful students, all of them are able to spend lavishly on education, all of them maintain a low dropout rate. These metrics account, collectively, for 82.5 percent of an institution’s “grade.” What puts one school above another at the top end, then, is what remains: the small stuff. Of the remaining 17.5 percentage points, 12.5 are awarded for selectivity — the proportion of students the school accepts versus turns down — and the percentage of alumni who give above a certain threshold.

The result of this system is a perverse set of incentives for top colleges who want to boost their rankings; namely, to put a great deal of effort into boosting selectivity and alumni giving participation rates and wringing the last few hundredths of a point out of the broad categories in which they perform well by default, which might make the difference between, say, 8th and 9th place. Thus, other excellent schools are beaten out.

Beyond simply being trivial, these incentives start to really matter when they begin to affect how schools make decisions about resource allocation. If the shot-callers start thinking less about where the money they have to allocate will do the most good for students and faculty and more about where it will get them points on the U.S. News rubric, opportunity costs start to build up. “We could put this money toward funding undergraduate research” thinks an administrator at University No. 7. “But we’re already beating University No. 6 in spending-per-student, and our alumni giving numbers could be improved, so we’d better spend it on a fundraising push instead.”

Of course, this is an oversimplification. No university would admit to making decisions this way, but there are documented cases of schools engaging in just this kind of thinking. Washington University in St. Louis, for example, was heavily criticized a number of years ago for rejecting students its admissions office considered likely to receive admission at more prestigious schools. Such students, they reasoned, were likely to turn down their offers of admission, therefore hurting their ranking metrics. Besides being fundamentally dishonest, this is a classic case of “quantity-before-quality” thinking by administrators — a willingness to sacrifice the quality of the academic experience for the sake of boosting stats.

So I’m glad to see that Penn has fallen a spot, not because I know for certain that they’ve rejected metrics-first thinking outright, but because it’s at least an indication that if they are trying to sacrifice academic quality to the gods of U.S. News, they’re not doing it very well. Maybe the brass deserves more credit than that, maybe less. From my point of view, however, things are at least headed in the right direction.

ALEC WARD is a College junior from Washington, D.C., studying history. His email address is alecward@sas.upenn.edu. Follow him on Twitter @TalkBackWard. “Talking Backward” appears every other Wednesday.

Please note All comments are eligible for publication in The Daily Pennsylvanian.