Over at her place Jenny D. asks what’s to worry about with this growth model experiment that Earth Mother wants to conduct. Well, Eduwonk and others are frequently criticized for being Luddites for raising concerns about the rush to change policy in this area. That said, while some innovation here is good, here are four things worth worrying about:
1) While many NCLB critics grumble that its accountability system only measures “status”, that is where a student is at a point in time, what they neglect to tell you is that that very few states have the ability to track individual students from year-to-year right now. So, while status is not ideal, a rolling average of how 4th-graders, for instance, do over three years does tell you something worth knowing in the meantime.*
2) In order to truly measure growth over time a state’s measurement system needs to have vertical alignment, meaning able to have a scaled measurement system so that achievement can be compared across grades. Not only is this challenging, but again a lot of states are a long way from here in terms of their standards and assessments and also again their ability track and analyze data. (This is different than the issue of vertical alignment in policy which is about whether a state’s pre-k to college systems are in alignment.)
3) Politically, any system predicated on measuring growth or relative progress begs the serious question of who gets to decide how much growth is enough and on what basis do they make those decisions? For all its problems NCLB’s current approach mitigates this by creating a common benchmark for all kids regardless of race and income. Some argue that’s unfair, but because it’s going to be the high-poverty schools most likely to be using “growth” as a way out from under NCLB’s current requirements, the potential for diminished expectations for such schools is very real. It’s not by coincidence that the loudest cheering for this was from those representing the schools and the most concern from civil rights groups. If indeed minorities are well served by the current system then there is nothing to worry about here…
4) Obviously everyone involved should endeavor to get the measurement as precise and reliable as possible but it’s certainly debatable whether the core problem here is measurement or politics. Assume for a moment that policymakers, statisticians, and psychometric experts had managed to agree on a system they all felt was valid and defensible and could accurately identify low-performing schools or schools needing improvement. Would it be any easier to intervene in those schools if it meant any real consequence for any adult? Would there really be much less pushback from all the organized groups? NCLB’s critics are more than happy to kick the can down the street a while by calling for more and more sophisticated measurement systems but it sure seems like the moment for real accountability is always just over the horizon.
None of these issues are insurmountable (though several also bear on the other accountability flavor of the month, value-added though there are additional complications there, too) and it’s possible to compromise on some issues and work through them. But, every compromise carries a consequence.
That said, trying this in a few carefully selected states is one thing — and would probably generate some useful information for policymakers and researchers. Trying it in ten states doesn’t make a lot of sense and raises the worrisome question of whether this provision will be the baseline for reauthorization of No Child whenever that happens because of the almost irresistible leveling-down that characterizes education policymaking decisions that involve any teeth.
*Shouldn’t one discussion in advance of reauthorization be whether the timetables for school improvement in NCLB should be modified? The flip side of the coin about whether the measurement system is right is whether the consequences that flow from it make sense. Without reducing the pressure for urgent change in seriously low-performing schools, isn’t there some play on that side of the policy? In other words, instead of worrying so much about the precision of measuring improvement, just give the schools that are not truly bad actors a little more time on the timetables within the law and add room for “growth” there. Alternatively states could triage schools to focus on the worst ones, perhaps more serious consequences but more of a time-lag between them, etc…it seems like something of a blue sky area that is being ignored in all the discussion about growth and value-added.
Update: Jenny D. has more plus a very smart comment.