Sin Of Commission?

As the dust settles on the Race to the Top selections a general consensus has emerged that again (pdf) there were problems with the scoring.   Not the sensational political tampering claims that some people are trying to allege, there is no evidence of that, but rather problems with the process.   Those problems are at once more mundane and a lot more far-reaching.  The Race to the Top is over for now but the problems have broader implications for federal grant competitions, especially because the Administration would like to shift toward using more competitive grants for some programs.

The rationale for moving to more competitive grants is that they involve more planning, more substantial amounts of money to make real changes, and that the competition pays dividends as competitors (even those that ultimately lose) vie to get ready to win.  The Race to the Top certainly illustrated this as did I3 and several of the other competitions running this year, including smaller below-the-radar ones.  And as long as the priorities of the competitions are well-constructed there is no reason that competitive grants can’t also serve the equity goals.  You wouldn’t want to allocate all aid this way, but on some big programs, such as Title II teacher quality funds (pdf), it makes a lot of sense.

However moving in this direction means that the competitive grant process must be reliable.  For example, scores should align to program goals and reviewers should be able to separate rhetoric from actionable proposals.  And there absolutely must be consistency among reviewers so it’s not a game of bingo for applicants.  In the case of Race to the Top, while it wasn’t a disaster there were enough problems that some people favorably inclined toward using more competitive grants are now asking if the federal government, with all the political and substantive constraints upon it, can really run a reliable high-stakes competition. (The back and forth around the social innovation fund isn’t helping either).

Meanwhile, you’re already hearing a lot of concern that because pretty much everyone with deep expertise on assessments was conflicted because of work with various states and vendors that the review pool is there is sub-optimal.  That was certainly the case on Race to the Top.   Watching some of the video interviews is discomforting as were some of the questions reviewers had on relatively basic issues.  Surely there is a better way to mitigate conflicts of interest but also engage people with deep knowledge of the work.  For my part I helped several state teams prep for their RTT live interviews and the feedback after their actual presentations was that the prep teams were much harder on them around the guts of the applications and the connective tissue that really makes plans like this rise or fall.  I heard the same from other prep projects.  That’s not encouraging.  Especially with hundreds of millions of  public dollars on the line.  Likewise, the actual reviewer comments and scoring variances in Round 1 and Round 2 don’t always inspire confidence, to put it gently.

It would be easy to say the problem was people at the Department of Education and a different team would have avoided these problems.  I don’t think that’s the case.  Rather, the competitions are thoughtfully designed but constrained by a flawed process (and of course a tortuous ‘gotcha’ politics).

That’s why Secretary Duncan must move quickly to head these problems off at the pass.   The best way for him to do so would be to convene a commission or Secretary’s technical working group to study and report on what can be learned from the federal competitions so far and, more importantly, what can be learned from other high-stakes competitions in the public and non-governmental sectors.  There are other fields, for instance, where a small subset of experts have to at once make decisions and manage conflicts of interest.  What are best practices there?  What are best practices for ensuring reliability among and between reviewers?  What aspects of current federal grant making policy (which was really not designed for high-stakes competitions like these) should be changed?  Is more training needed, and if so what kind?  What else has to change if substantial amounts of federal aid are to be allocated this way?

Granted (ha ha) commissions are generally considered the place to go when you don’t want much to happen or want to punt an issue.  But Secretary Duncan has proven clever at leveraging issues in creative ways and just by putting his brand on it and sanctioning a candid review and study he could make such a process meaningful.  Substantive benefits aside, such a project would also help the Secretary make the case for moving in this direction.

Bottom line:  The fallout here extends beyond states like CO, IL, and LA and it behooves the Secretary to get in front of it.  Not in a way that invites pointless recriminations, but rather in a forward looking way that improves future initiatives.

Standing Disc: Bellwether personnel, including me, were involved in advising a number of states, winners and losers, about policy and strategy but had no direct interest in a specific outcome for the states mentioned here.

Good Reading – Now With More Polls!

Interesting paper from Bridgespan on next generation learning initiatives. Tom Friedman hearts Waiting for Superman.  And an op-ed from the President of Strayer lays out the contours of the debate over for-profit higher education and accountability.  If nothing else that debate is going to finance the college tuition of a lot of lobbyists’ children.

Plus Gallup/PDK poll v. Ed Next poll (pdf) again this year.   Release event for Gallup/PDK today, it’s improving a lot to their credit and has the Gallup brand but Ed Next still has the mojo.

Update: More on the polls.  First, while Gallup/PDK find low-approval for President Obama on education and a drop in support it does not seem keyed to his policies.  Why?  When Gallup/PDK or Ed Next digs down on a host of those you find bipartisan support.  Rather, the education approval numbers seem to be referred pain from his overall low-approval numbers. If you only look at the (in my view badly worded) question on turnarounds of course it will look different.  Update: Unfair to only pick on Ed Week, others bit on this, too.

Second, although the number has remained roughly consistent over time, about 4 in 10 public school parents say they’d change schools if they could.  From a pure loyalty and market share perspective that should be a troubling number for the public school establishment but instead they take comfort from the 60 percent.

Third, my friends in the charter world are gaga over the record high support for charters in the PDK/Gallup poll.   But I wouldn’t pop the champagne just yet, that support seems fragile.  Indeed, check out Ed Next’s poll and some other polls that are out there.  The public may like charters but it’s unclear they have any idea what one is!  In Ed Next even 1 in 4 teachers said charters charge tuition.

RTT Day After

I had the craziest dream last night, Louisiana, a state that is a leader on all the things that the administration says are priorities didn’t get Race to the Top funding…oh wait…

Anyway, New York never disappoints, the Patterson presser is one for the ages.  ‘Race to the cock?’  What the hell?

Big takeaways beyond the RTT issues below, are that the odds of seeing consistent and deep change across all Race to the Top winners got a lot longer with this round of selections.   But the two fundamental questions basically remain the same and can’t be answered yet:  How durable will the many RTT-inspired policy changes prove to be and will those changes actually improve student learning?

Over at Fordham Mike Petrilli’s doing the full-Chicken Little again, “disastrous” and “complete lack of political courage” he shrieks!  That’s over the top.  You are not hearing outrage across the board, we’re talking about a few states.  But, he does hit important themes in this blog post and admirably acknowledges that he would have criticized the administration anyway, even if they had made the exact decision he’s now urging. In other words, the administration was damned if they did or didn’t here in this environment.  So, even if you’re disappointed about CO and LA and some of the outcomes that ought to be acknowledged – they were in a tight spot.  And don’t miss Checker Finn’s very even-handed take on the outcomes.

A second thing worth noting is that some of the leaders in states that didn’t win now have exposure because of problems with this review process.   For instance state ed chief Paul Pastorek in LA and his team, Dwight Jones and the CO team, and Audrey Soglin, the state teachers union leader from Illinois.  All of them took risks based on the avowed goals and rules of the competition.  But if anything their stature should be higher than it was previously and especially relative to some other states.   You can find plenty of people who didn’t like parts of the applications from states like IL, LA, or CO but you’ll be  hard pressed to find anyone who follows this closely and is familiar with the applications and doesn’t think those states were superior – based on the avowed goals and rules of the competition – to some winners.

Biggest fake RTT trend so far?   Geography. Second place:  Word selection. Most prescient call: The New Teacher Project (pdf).

Standing Disc: Bellwether personnel, including me, were involved in advising a number of states, winners and losers, about policy and strategy but had no direct interest in a specific outcome for the states mentioned here.

RTT – Updated

CongratsWinners list is public now:  DC, FL, GA HI, MD, MA, NY, NC, OH, RI.

Reminder: Round II finalists were:  AZ, CA, CO, DC, FL, GA, HI, IL, KY, LA, MD, MA, NJ, NY, NC, OH, PA, RI, SC

Fallout: This list is causing some raised eyebrows already.   Keep an eye out for questions questions about LA and CO and the relative strengths of their apps (and relative controversy around issues like teacher evaluation and holding schools of ed accountable) and also how New York went from not meeting the basic eligibility for the competition to being a winner.

And keep an eye on DC, FL, GA, HI MD, MA, NY, OH, and RI, which all have governors (mayor in DC) races ongoing right now.   Rothenberg ratings here.  Implementation implications.

Update: To be clear, we have to wait until all the scores are out but it seems that again the problem was not a thumb on the scale but rather a thumb off the scale and reviewers that didn’t reflect the administration’s avowed reform priorities.

Update II: Big winner is TFA?   Obvious way: Teach For America is mentioned in all 12 winning applications from Round 1 and Round 2.   Less obvious way:  TFA has been pushing back on the move to make Title II teacher quality funding more based on competitive grants, they worry about an uneven grant process, going to be harder to argue with them now…

Update III: John Bailey spots some assessment competition implications, too.

Standing Disc: Bellwether personnel, including me, were involved in advising a number of states, winners and losers, about policy and strategy.

Look Around

Isn’t all this concern about the mixed quality of school turnaround providers somewhat decontextualized?  Is there any issue in education from publishing to professional development to online and tech where there is not a high variance in quality among providers and an inability in many states and school districts to make good quality-based decisions (as well as a chronic lack of tools for doing so)?  In other words, any big dollop of federal money would shine a light on this problem regardless of the specific issue it was targeted to, we just happen to be focused on turnarounds right now.

Related, I’m less concerned about the federal turnaround grants moving too fast or slow (seems like some of both overall) than about what’s actually happening as a result, which so far seems a mixed bag at best.

Update: Justin Cohen on the same issue.

Odds And Ends

Two Johns have two takes on the LAT value-added imbroglio worth checking out.   John Fensterwald looks at the school district’s positioning, and John Merrow comes out in favor of naming names.

For all the political sniping, hardly anyone noticed that Democrats for Education Reform and the AFT ran a joint ad for Michael Bennet in Colorado.

BW’s Sara Mead does a sit down on Ezra Klein’s blog to discuss early childhood education.

Meanwhile Michelle Rhee gets the Jay Mathews endorsement in the upcoming DC primary. See also this on the mayoral race.  Rhee has better approval ratings than the President!

Don’t miss SEC v. New Jersey on pensions if you follow that issue.

And from Politico’s Obama-teachers story is this:

Nationwide outrage among teachers exploded in March when both Obama and Duncan justified the mass firings of educators at a failing Rhode Island school. (Teachers ultimately kept their jobs in a concession deal.)

“The administration has strong reformist credentials, but this went way too far for many people,” said Richard Kahlenberg, a senior fellow at the Century Foundation who studies teachers unions. “I think the average American who worries about getting laid off unfairly could relate to the teachers.”

Just to be clear, teachers ultimately kept their jobs in the deal the school district wanted at the beginning before all the teachers were fired. The only difference was the months lost to face-saving rather than planning.   So (a) there was no concession and (b) contra Kahlenberg’s plea for solidarity polls at the time were supportive of the district and Obama.   The average teachers’ union member may have been pissed but that’s not the same thing…except perhaps at Century?

Meanwhile, WaPo’s usually sharp Birnbaum falls for the “dueling” studies cop-out on TFA research.   In his defense, a straightforward look at the research would make the article pretty pointless.  More generally, isn’t it time to flip the script on this and ask what it means for traditional teacher preparation programs when teachers coming through routes like TNTP training programs and TFA do as well at substantially less cost to taxpayers and candidates? That would be an interesting article with some real implications for where policymaking is going on that issue.  (And, disc, I’m on the boards of two ed schools – UVA and Harvard – so I’m not hostile to training and prep).

Friday Fish Porn! Father Of The Bride Edition!

DSCN0814Journalist and author Richard Whitmire has guestblogged on Eduwonk and been featured in past fish porn posts. And he and I write together from time to time about education issues.

He also has two terrific daughters (one of whom is getting married in a few weeks) and the other who works with Stand for Children in Portland, Oregon.   Bellwether also works with Stand on policy issues.  So, early this summer everything came together for a day of fishing on Washington’s Clackamas River.   Here’s Whitmire with a sea run cutthroat trout.

From last week’s guest fish porn, here’s some bonus triggerfish trivia.

LA Confidential?

AFT’s Randi Weingarten attempts to strike a middle ground in the LAT teacher value-added debate. Pretty reasonable and admirable she didn’t do the easy wrong thing there and just throw-in with the naysayers.   A lot of analysis based on the 6k teachers in this database would be valuable and a real service by the LAT, but what exactly is the benefit of naming all the teachers?  It would also be great to know in a year, two years, etc…what’s been done about the lowest performers – again, the district has some exposure here- but that, too, can be done absent a show trial.

But at the end of the story there is a headscratcher and yoga-like rhetorical stretch: Weingarten compares a teacher with low-value added, who is nonetheless beloved, to Shirley Sherrod the falsely-maligned former USDA official because both were allegedly victims of partial information.  That’s indisputably true in the Sherrod case, but illustrates the problem in this instance.   Beloved and effective are not axiomatically the same thing.  And Weingarten’s claim that this might be such a good teacher that it doesn’t show up on standardized tests is the sort of faux-wisdom that keeps this field so screwed-up.  I’m not saying the value-add scores indicate anything definitive about this particular teacher one way or the other but I am saying we should be more open to the possibility. And principals with experience with value-add will tell you the exact same thing:  It challenges some preconceptions and they learn from it and sometimes probe deeper.

What worries me about this whole exercise: If the LAT reporters/editors on this story didn’t have the chops to call BS on that one, do you really want them publicly impugning teachers with this data?  I don’t.