Seebach: This week's rankings of the states intriguing but flawed

Saturday, December 16, 2006

There is no end to the making of lists that rank the states according to some variables or other that the list maker thinks are important. Two quite different ones came across my desk this week, both interesting but ultimately unsatisfying.

One is from the Cato Institute, which has developed the "Cato Education Market Index." The purpose of the index "is to rate existing school systems on the basis of how closely they approximate true free markets (we call this a market rating) and to rate education policy proposals on their conduciveness to the growth of markets (a policy rating)."

The full report, written by Andrew Coulson, is extremely complex and I won't even try to explain the methodology. In fact, I'm pretty sure I don't understand it. As Coulson notes, rankings of this kind are usually calculated as sums or averages of their components (a practice I commonly hold up to ridicule, although sometimes it makes sense). That takes no account of the interactions between components, which is clearly not true for most of the variables that could reasonably be included in an index.

So, he says, "This interrelation of the producer and consumer components of free education markets implies that our index is better calculated as a product than as a sum or average."

I guess that's plausible; I just have no experience with it, and no intuitive sense of how it works. As for the results, the scores are all quite low - as you'd expect, given that there's essentially no free market in education. On market ratings, Wisconsin and Connecticut are tied at a score of 26 (on a scale of 1 to 100) and Colorado is 10, in a tie for 38th place with Oregon, North Dakota and Mississippi. On policy ratings, the leaders are Texas and Wisconsin, and Colorado is in a 37th place tie with Iowa.

Does that sound right? I have no idea. And unfortunately, it probably doesn't matter, because the constituency for a true free market in education could probably meet in Cato's conference room, if it has one, while the entrenched interests favoring the education system we have now are powerful and numerous.

If states are ranked by something a lot of people care about (spending on schools, say), then a lot of people will use the ranking to advance their own agendas, so it will have influence. People still quote to me Education Week's ranking of Colorado as 48th in the nation on school spending, though that was years ago and it wasn't actual spending that was ranked, but EdWeek's own mix of oddly weighted variables.

The other report I saw this week was the Education Watch 2006 State Summaries, released by the Education Trust. It makes no attempt to do an overall ranking, which is wise, but for specific indicators - the size of the Latino-white gap in fourth-grade reading on the National Assessment of Educational Progress, say - it shows bar graphs rank-ordering the states so you can easily see where your state falls. Loads of useful information here, and lots more on their Web site.

The Ed Trust focuses on equity issues. They say, "The information in these reports reveals how far we have to go to ensure that every young American has access to high-quality education."

But their undoubted passion for equity leads them to make occasional dubious assertions. Most striking was the claim about Advanced Placement courses, that "In a system where all students have equal access to these opportunities, the percentage of test takers by race and ethnicity would be proportional to their representation in public K-12 enrollment."

No. It wouldn't. Students with the same opportunities might make different choices about whether to take advantage of them, and there's no reason to assume that strict proportionality will necessarily result, though sometimes it might. If one school in a district offers many AP courses, and another few or none, that is a difference of opportunity. But if students in the same school enroll in different proportions, the school can encourage and cajole those who don't enroll, but it cannot coerce them.

Consider Lowell High School, an examination school in San Francisco. As it happens, the percentage of whites at Lowell is the same as in the district as a whole, 15 percent. For Filipinos, too, it's the same, 6 percent. But for Chinese, Japanese and Koreans, the percentage for the whole district is 34 percent, while at Lowell Asians are almost twice that, 67 percent. Would anyone credit the claim that the reason white students are half as likely to qualify for Lowell as Asians is that they had lesser educational opportunities in earlier grades?

Daria Hall, an analyst with the Ed Trust, defended the comment. She agreed that poor and minority children often enter school already behind. But, she said, "the schools do nothing to ameliorate that problem; instead, they exacerbate it."

Now, on that we agree.

Linda Seebach is an editorial writer for the News. She can be reached by telephone at (303) 954-2519 or by e-mail at seebach@RockyMountainNews.com.