A few years ago Sara Mead and I wrote a paper (pdf) criticizing the “Challenge Index” that The Washington Post and Newsweek use to rate high schools. We debated the issue with the index’s creator, Jay Mathews, but in the end his position boiled down to, ‘if you don’t like this system, build a better one.’ So along with US News and S&P that’s what we did. Methods here (pdf).
Edweek’s flagship Quality Counts, just released yesterday, could be on a similar trajectory. As Macke Raymond (who among other roles chairs ES’ board) points out the metrics used by Education Week paint a distorted picture of school quality (I’d also quibble with other aspects like how EW accounts for accountability). Raymond’s not the first to make that point but she is the first to start to slice the data in different ways in another publication: Ed Next. We could be seeing another rankings competition starting?
With high schools our concern was that some really lousy high schools that did well by a few kids and badly by most were able to hit the board in the Challenge Index. Here the problem is somewhat different. But the outcome with Quality Counts in some ways matters even more because it drives politics and policy. In states that get really good grades because of the “chance for success” index it has a chilling effect on reform. ‘Why should we fix our schools’, people argue, ‘Ed Week says they’re great!’ Reform is contentious enough without creating that false sense of security.