Article in American Journal of Education Questions Validity of Published Rankings of Graduate Schools
by Joe Savrock (May 2010)
UNIVERSITY PARK, Pa. – Each year, university administrators, faculty, and prospective students await with great anticipation the release of graduate school rankings by US News & World Report. These rankings, based on reputational assessments made by senior administrators at peer universities and liberal arts colleges, are recognized as the paramount measure of the nation’s best graduate school programs.
The annual rankings are called into question in a recently published article in the American Journal of Education (AJE). Researchers Michael N. Bastedo (University of Michigan) and Nicholas A. Bowman (University of Notre Dame) analyzed published college rankings from 1995 to 2006 in order to explore their institutional effects on the reputation of academic programs. The researchers found that rankings have a significant impact on future peer assessments.
Their article, “U.S. News & World Report College Rankings: Modeling Institutional Effects on Organizational Reputation,” appears in AJE’s February 2010 issue. AJE’s editorial home is the Department of Education Policy Studies in Penn State’s College of Education.
Bastedo and Bowman found evidence that the administrators of peer institutions, when making their annual assessments of reputation, can be swayed by rankings of prior years.
“The article reveals an institutional repetition effect, whereby rankings tend to replicate themselves,” said Gerald LeTendre, AJE editor and department head of Education Policy Studies. “In other words, current rankings are influenced by how institutions have been ranked in the past.”
The article suggests that a significant change in a school’s year-to-year ranked position sometimes occurs even when there have been no significant changes in the school’s organizational quality and performance. The consequences of placement in the rankings can greatly outdistance the miniscule, seemingly innocuous changes in academic quality.
“Significant changes in the quality of a graduate program over time don’t have as much of an effect in the rankings as perhaps they should,” noted LeTendre. “At the same time, small differences in performance indicators that may or may not be significant are used to generate higher or lower ranks.”
Bastedo and Bowman, in their article, write, “Thus higher education becomes a winner-take-all market where marginal differences in performance lead to large differences in reputation…. When prestige is academic currency, the result is a ‘positional arms race,’ where colleges spend significant resources to attract students who differ only marginally on indicators of quality.”
LeTendre further notes that, “Other published rankings—those that measure affordability, research productivity, and other qualities—get underplayed in light of the US News rankings.”
The complete article is available online (PDF, 154 KB).
