Mathematica just released a study that did not find learning gains or improvements in teacher retention as a result of the Teacher Advancement Program (TAP)* in Chicago. The initiative was funded via the federal Teacher Incentive Fund. There is plenty of bloggy noise about this so just five quick observations mostly about context:
1) This is just one study (and a small one, eight schools each year, that also had some attrition as schools were closed) of TAP, so the rush to judgment is lacking in context. Overall the evidence on TAP is mixed but encouraging and the big challenge has been controlling for selection effects (teachers choose to participate), which Mathematica** sought to do through their design here. TAP is doing a second study to see what’s going on in Chicago, stay tuned for that. Hopefully some lessons for Chicago but also more generally.
2) TAP is a lot more than merit pay, it’s about creating a career ladder for teachers that creates opportunities for new responsibilities and opportunities that do not mean a teacher has to leave classroom teaching. That’s why it’s supported by organizations like the American Federation of Teachers. So the rhetoric about how this is some sort of verdict on merit pay is uninformed.
3) This is just one Teacher Incentive Fund location so everyone should slow down with the rush to a verdict on that initiative. And anyone who thinks the field knows how to do any sort of incentive pay has not been paying attention. What’s known is that the current system of allocating about 4/5ths of the $600-plus billion we spend annually on K-12 schools is not linked with outcomes and creates some perverse incentives. How to fix that is going to require a lot of innovation, experimentation, and failure.
4) As with Reading First and other initiatives, the apparent glee and celebration when things don’t work is one of the most depressing aspects of education policy today. Something went wrong with this initiative in Chicago, to their credit TAP wants to figure out what it is rather than attack the study, make excuses, etc…and yet what you get from the bleachers is some cheering and crowing. Really?
5) If this was not Chicago would it be getting as much attention?
*Bellwether works with TAP and its parent organization the National Institute for Effective Teaching.
**Mathematica continues to come up with creative evaluation strategies on issues (eg. teacher effectiveness) that previously were assumed to be impervious to this sort of evaluation.