Rick Hess takes a look at the recent MET data release from the Gates Foundation. Good points and worth reading. The more interesting conversation happening behind the scenes among some analysts – and spilling into the public some – is whether Gates is underselling some of what the MET data says in terms of value-added measures. I think they are to some extent, but not out of any nefarious scheme. Rather, they clearly think that the best way to progress on this issue is achieving broad teacher buy-in and are crafting a strategy accordingly. Call that the long-view approach, modest progress now in exchange for big shifts later. Others argue for more robust policies now, especially with regard to persistently poor performers, and that could be supported based on the Gates evidence and other data. Reasonable people can disagree on which approach will have more payoff for kids in a decade or two but we should acknowledge there are trade-offs and risks to both strategies. Not unrelated, a few years ago I noted that litigation is likely around value-added when it starts being used in consequential ways, keep an eye on that.
Marc Tucker with a thoughtful look at Common Core. Key NCLB player Sandy Kress takes a look at that law’s 11th birthday.
Roll public schools? Dan Ross discusses what lessons Alabama’s Nick Saban offers managers in public education. Ed Week reports on some important new research on the Moving to Opportunity Program. Mixed results, disappointing academic results. Interesting implications for both school improvement strategies and economic integration conversation.
Education fellowships at the Southern Education Foundation. And a great opportunity in TN – First to the Top Fellowships.
Big get: USDOE Asst. Secretary and Arne Duncan confidante Peter Cunningham is joining Whiteboard Advisors.
Loved Sandy Kress alternative reality over at DropNation.
Reformer cant help but make stuff up.
Hereeeee’s Sandy:
And the facts, ma’am:
I cited data before and after 1999 because, as Rick Hanushek showed in his research, half the states were in consequential accountability by 1999. My claim for gains in the last decade is not due just to NCLB but to the standards-based reform movement of which it was the high point. (Something tells me Marlowe doesn’t support the policies that swept the nation starting in the mid-90s any more than he likes NCLB.)
The reason I use the LTT data is that both sides of this debate want to grab the unusual gains around 2002 and 2003 for their side of pre-post. I suspect Marlowe knows that. Most serious observers are concerned enough about that spike to say it ought to be left out of analysis, if possible.
Finally, I notice that Marlowe did not want to list the math data at all. Hmmm.
Mr. Biddle’s cheque hasn’t clear yet , so I haven’t done math.
Hmmmm.
And if the best you can come with is that math a d reading instruction was the same before and after NCLB,
Hmmmm.
In Maryland, things are “sig ificantly” different 20 years ago and the way they have been for the past decade. Instruction methods, content, assessment, the dropping of science in primary grades.
Hmmmm.
And I m surprised you didn’t point out how NCLB has very very good for Asian students.
The bottom line is, the reformers themselves caused the current conflict and slowdown in reformation efforts. First, they should not have erected the crumbling edifice of reform on the work of fiction entitled, “A Nation at Risk.” Second, and even more important, they alienated almost from the get-go the one group they needed to push their reforms–teachers.
No, it wasn’t the unions that stood in your way as much as rank and file classroom teachers making their own everyday decisions about their own professional lives and work. Teachers were told they were the most important link in a way that sounded kind of empowering but really masked a cynical drive to find blame that served the political and ideological aspects of reform. The big money from the technology and financial sectors of the country have overpowered the debates and have funded more experiments in social engineering than vital educational research.
The research (not out of think tanks) that has been done has provided mixed reviews of various efforts at reform from the classroom to curriculum to large scale management. Without clear signals to reshape the system, there was bound to be only modest and eventually waning support from teachers to alter their practices or take on new roles. The modest rise in NAEP scores cannot be attributed to NCLB without more solid research; the rise had started before 2001. Long-term SAT scores have remained rather flat suggesting that any rise in NAEP may be found in an examination of the exams and scoring.
OK, Kress
Here you go:
Grade 4
Math
Black
1990-2003 +28 (188-216)
2003-2011 +8 (216-224)
Hispanic
1990-2003 +22 (200-222)
2003-2011 +7 (222-229)
And, Kress, here’s 8th grade:
Grade 8
Math
Black
1990-2003 + 15(237-252)
2003-2011 + 10( 252-262)
Hispanic
1990-2003 + 13(246-259)
2003-2011 + 11(259-270)
Asian
1990-2003 +16 (275-291)
2003-2011 +12 (291-303)
Hmmmmmmmmm.
8th grade Math
Black
2005 255
2007 260
2009 261
2011 262
So, Kress, granting you a more favorable take, we see that NCLB may have made scores jump , but by a rather amount.
This reminds me a story my friend Alex would tell:
You guys are all idiots. Ever heard of a difference-in-difference model? If you really want to know if NCLB/accountability had a positive CAUSAL impact on achievement you’d read Thomas Dee’s piece in Journal of Policy Analysis and Management. But then that would probably require more than addition and subtraction skills. #intellectualLIGHTWEIGHTS
Hi DaProfessor,
Sorry, but it doesn’t work out that way in a quasi-experimental design. Causality is suspect. While such research can guide future efforts, one should not rely on the methodology Dee describes for final conclusions concerning causality. It’s not me saying that. Check it out.
Case in point. Dee says, “In particular, by focusing on low-stakes NAEP scores rather than high-stakes state assessments, our analysis can circumvent concerns about the construct validity of state assessment scores.” Sure it can. Right. Lacking any footnotes for this assertion, we must take Dee on faith. Low-stakes for whom? And even if Dee deigns to tell us that, we would still be left wondering just how it is children perceive the relative value of standardized exams.
What does accountability mean? How is it defined by Dee? Hard to say. Here’s the best I found, “For example, Hanushek and Raymond (2001) write that accountability policies are “premised on an assumption that a focus on student outcomes will lead to behavioral changes by students, teachers, and schools to align with the performance goals of the system” and that “explicit incentives… will lead to innovation, efficiency, and fixes to any observed performance problems.”” A focus on outcomes. That’s it? Focus? What in the world does that mean?
Do you seriously expect me or anyone to take seriously a work of scholarship in which fundamental definitions are not addressed? Hey, I like that Dee supports everything I have said here. Dee helps me make my argument. What I don’t like is that Dee is not the exemplar of support for NCLB DaProfessor claims.
From Dee, “For example, in the presence of a high-stakes performance
threshold, schools may reallocate instructional effort away from high and low-performing students and towards the “bubble kids” who are most likely, with additional attention, to meet the proficiency standard (e.g., Neal and Schanzenbach, in press).” OK, I have my own notion of what it means to “reallocate instructional effort” but I’m not sure it is the same as for the authors. Bubble kids? Look, I know full-well about something “in press;” what is sketchy is when an author doesn’t even care to offer a working definition in place of the hoped-for “in press.” You cannot make a substantive assertion of value and attribute your evidence as ‘something that we promise to show you at some point.’ I know, I know, researchers do that cute trick more often than we care to admit. But dammit, you have to admit you’re doing so and that future research may or may not flesh-out research assumptions.
Why are economists entitled to inveigh upon all of life? It didn’t used to be this way. Why are economists so entranced with education? Why are so many hedge fund managers and high technology firms invested in education reform? I’m through with this waste of time. It’s a very nice-looking study that is riddled with wormholes. Go fish.
Nice trick, Marlowe. Create a pre- period of 13 years and compare it to a post- period of 8 years. You take that indefensible step because you want and need to grab main NAEP ‘s 2003 gains for your pre- period. Without doing that, your numbers crumble.
2003 was near the peak of the adoption by states and the nation as a whole of consequential accountability policies. I’m not going to assert certain confidence in causality. No one can. But there was nothing on the policy or any other landscape that warrants counting the improvement after 2000 in a pre-accountability period. Nothing. Indeed all the evidence is to the contrary, that the direction had been pervasively toward accountability and was reaching its high point in the mid-2000s.
I chose the Long Term trend data because it had two equal periods, it had the year when accountability reached a tipping point as a boundary between the periods, and it didn’t have the unusual bump in 2003 to resolve.
One thing’s for sure: opposition to accountability has risen significantly in the last few years. If that grows, it will be interesting to see what happens in the 10 year period beginning in 2009. May the good Lord bring us all back together to look at student achievement trends in the post post decade.
DaProfessor,
I am so glad I made your feel better.
Are you going to Belwether’s LoveFest with Andy Smarick in 11 days in DC?
IF so, I can give you the opportunity to feel even better.
If not, just Call Me Maybe when your spouse/lover has had enough of you and told you to stuff it.
Kress,
the effect of NCLB accountability in any classroom began in 2003. NAEP from that year would have shown minuscule influence of NCLB accountability.
In Maryland, that was the first year for their NLCB test, the MSA.
Play with the numbers whatever way you want to make them show what you want, and I will do the same in return.
Long live our King
25 years of Hunger, Misery and War</i?
@JeffMiller
Never said Dee’s study was without flaw. Never said it was evidence that high stakes accountability is the best policy out there. All I claimed was a DnD analysis in a top tier peer reviewed journal is a better starting point for debate than a couple of bozos doing addition and subtraction in the comment threads of a blog. Let’s get serious here intellectually shall we.
P.S. Do you have anything published in a scholarly journal with an impact factor score as high as JPAM?
@DaProfessor, I gave you a detailed and serious response. The answer to your question is no. But you know, I find it funny so many researchers without education credentials feel entitled to inveigh upon educational matters with their fancy math and non sequitur retorts. Speaking of lacking critical credentials, here comes Mr. Kress now…
@Sandy, “I’m not going to assert certain confidence in causality. No one can.” But you did, in attempting to remove confounding variables. What is your operational definition of this “accountability reached a tipping point as a boundary”? What sort of accountability, what defined the tipping point? I think that maybe your total and utter lack of experience in the classroom leads you to erroneous assumptions and invalid conclusions. Even Petrilli disagrees with you and supports Marlowe. And he, like you, seems to be someone who worships at the holy church of accountability as actually meaning anything. http://www.edexcellence.net/commentary/education-gadfly-daily/flypaper/2011/fact-checking-sandy-kress.html
“One thing’s for sure: opposition to accountability has risen significantly in the last few years.” No, it’s not for sure. Your statement is an assertion, an interpretation of the data. Nothing more, unless you can show otherwise. Maybe it’s not even ‘accountability’ that teachers oppose; maybe it’s the quality of the reform measures proposed by the elite that simply spells FAIL.
But hey, keep trying to make your dog hunt and I will offer, ‘Proceed, Mr. Kress.’ Reformers just keep digging that hole deeper.
So, DaProfessor, you won’t be at the Andy Smarick LoveFest in DC?
I feel sad, with the tears of a clown. QQ
Lastly DuProfesor, it is not surprising that your arguments are reduced to racist name calling.
http://en.m.wikipedia.org/wiki/Bozo_people
http://hanushek.stanford.edu/sites/default/files/publications/hanushek%2Braymond.2005%20jpam%2024-2.pdf
To Miller – Hanushek and Raymond found in their research that there was a dramatic turn to consequential accountability between 1996 and 2000. Twelve states had such policies in 1996; 39 states did by 2000. This is the basis for my asserting that the year of 1999 in the Long Term data was a reasonable demarcation point for pre and post.
Listen, pal: each and very one of us is trying to make sense of very complicated issues. In all the abundant research I’ve read, I’ve never seen you mentioned once. So, cool your jets on credentials, skills, and experience.
It is nonetheless not possible to investigate the impact of NCLB directly. …the law has many facets, making it hard to isolate the effects of any single one. Finally, the common pace of NCLB implementation across the states eliminates any status quo alternatives for comparison.
An apology to Miller: I looked you up after my last post. While it is true I have never seen or heard of any significant research of yours on these big issues, I do see your past brief past position at Phoenix and your business in methodology. So, please accept this as a correction.
To Marlowe – your last post indeed raises valid concerns. We make hypotheses because we are all trying to make judgments about policies and their effectiveness. Whatever position we take on these matters, we want to try to assess and judge. Solid research takes us some distance. Reason and judgment carry us somewhat further. But, at the end of the day, real and conclusive proof is unlikely to find for the reasons you mention and others.
Though we’ve hit each other pretty hard over the past couple of days, I actually think there’s been light from both sides – in addition to the heat.
I’d like to delve more into Maryland on another day.
Sorry Sandy, no worries. I have a REALLY common name. Unless of course this is your idea of snark, in which case, well-played sir. U of Phoenix represents a blight upon higher ed and a drain upon federal student loans.
I simply want to know what accountability means. I am but a humble ABD, despite my advanced years and experience between higher ed and K-12. My most recent publication was in October, 2012. If you don’t troll the professional literature of K-12 instruction, I can understand why you missed me. Me, I tend to consider the considered thoughts of even the most lowly of unpublished kindergarten teachers worthy of respect.
From Hanushek and Raymond, “The cornerstone of current federal educational policy has been expansion of school
accountability based on measured student test performance.” But the test performance is situated within the assumptions the test designers make. There is no discussion of this important consideration. From page 25, “First, accountability as seen during the 1990s tended to help White achievement more than Black achievement.” Even if we assume the methodology and assumptions of what ‘accountability’ means as valid, the differential exposure to the variable of accountability begs a social interpretation. Need I explicate? Need I?
Listen pal, I get that we’re all trying to make sense of very complicated issues. I also get that some of us have political agendas and ideological fantasies. Capice? The only reason you bothered to respond to me or Marlowe is because we get under your skin–and while I can’t speak for Marlowe because I have no idea who s/he is, I will not sleep until those who mean to undermine the authority and professional experience of the classroom teacher are exposed. If you want to make changes to the system and you believe your econometrics are valid, you will have to make that fancy math accessible to rank and file teachers for interpretation and validation. Can you tell a kindergarten teacher or high school chemistry teacher with a BA, in simple English, how Hanushek and Raymond went about their research, their methodology, their mathematical analysis?
This is called teaching, Sandy. The job of the teacher is to break down and make accessible complex ideas to younger and less experienced students. That is YOUR job as well as mine. You can’t just tell people what to believe. I mean, hey, you Bushies did a good job of convincing people Saddam did have WMDs despite counterfactual evidence and now you pretend to tell ME how to perceive reality? Your NCLB/accountability data is rife with unproven assumptions and tortured statistical analyses. You know it. I know you know it.
So, I’m throwing it down. Translate Hanushek or whatever study you choose that purports to show a causal link between accountability before and after 2001 to higher student achievement on standardized tests (how you define accountability) into everyday speech that even kindergarten teachers or gym coaches or I will understand. One other thing: you DO understand don’t you, that the word accountability has both connotative and denotative dimensions? It’s not just some value-free econometric. It is loaded. Please consider that in your response.
“We make hypotheses because we are all trying to make judgments about policies and their effectiveness.” The trouble is that your hypotheses often end up as policy goals and initiatives without authorizing or waiting for research findings. “Solid research takes us some distance. Reason and judgment carry us somewhat further.” In the physical sciences, your assumption here would be completely invalidated. In the social sciences, you do indeed have room. Reason and judgment are by nature, subjective and thus open to discussion. But you see, education in this country is a contested terrain. It is political because our founders seem to have left education a matter of local concern rather than leaving it to professionals. I regard our founders blameless; they could not have understood how education is a matter best left to those with the expertise any more than the profession of medicine should be left to professionals.
Oh, snap.
Dear Sandy Kress
RE: Classroom Experience
I think it may be possible to analyze the professional/work effectiveness of a class of individuals without ever having been a member of said class. You do this for a living with educators. How do you create internal validation of your conclusions without personal experience? I mean, you’re not counting the number of protons leaving the Sun over a standard time period as a measure of the solar wind; you are trying to measure accountability presumably, of teachers, not students, right? The higher the test score, the more accountable the teacher is. Is that it? Can anyone have your job?
“We make hypotheses because we are all trying to make judgments about policies and their effectiveness.” No, you are not making hypotheses, you are making judgments. A judgment and an hypothesis are two different things. A hypothesis normally precedes an experiment which tests variables presented in a hypothesis. Only after an experiment has been performed and conducted by others may an estimation of effectiveness be offered. Look Kress, and others in policy, go ahead and run experiments but avoid making judgments without empirical evidence. And here I must repeat, “Solid research takes us some distance. Reason and judgment carry us somewhat further.” NO! If the research is solid, there is NO need for your “reason” or “judgement”. Do you get it??
I am actually grateful to the owner of this website who has shared this great article at at
this time.