As the dust settles on the Race to the Top selections a general consensus has emerged that again (pdf) there were problems with the scoring. Not the sensational political tampering claims that some people are trying to allege, there is no evidence of that, but rather problems with the process. Those problems are at once more mundane and a lot more far-reaching. The Race to the Top is over for now but the problems have broader implications for federal grant competitions, especially because the Administration would like to shift toward using more competitive grants for some programs.
The rationale for moving to more competitive grants is that they involve more planning, more substantial amounts of money to make real changes, and that the competition pays dividends as competitors (even those that ultimately lose) vie to get ready to win. The Race to the Top certainly illustrated this as did I3 and several of the other competitions running this year, including smaller below-the-radar ones. And as long as the priorities of the competitions are well-constructed there is no reason that competitive grants can’t also serve the equity goals. You wouldn’t want to allocate all aid this way, but on some big programs, such as Title II teacher quality funds (pdf), it makes a lot of sense.
However moving in this direction means that the competitive grant process must be reliable. For example, scores should align to program goals and reviewers should be able to separate rhetoric from actionable proposals. And there absolutely must be consistency among reviewers so it’s not a game of bingo for applicants. In the case of Race to the Top, while it wasn’t a disaster there were enough problems that some people favorably inclined toward using more competitive grants are now asking if the federal government, with all the political and substantive constraints upon it, can really run a reliable high-stakes competition. (The back and forth around the social innovation fund isn’t helping either).
Meanwhile, you’re already hearing a lot of concern that because pretty much everyone with deep expertise on assessments was conflicted because of work with various states and vendors that the review pool is there is sub-optimal. That was certainly the case on Race to the Top. Watching some of the video interviews is discomforting as were some of the questions reviewers had on relatively basic issues. Surely there is a better way to mitigate conflicts of interest but also engage people with deep knowledge of the work. For my part I helped several state teams prep for their RTT live interviews and the feedback after their actual presentations was that the prep teams were much harder on them around the guts of the applications and the connective tissue that really makes plans like this rise or fall. I heard the same from other prep projects. That’s not encouraging. Especially with hundreds of millions of public dollars on the line. Likewise, the actual reviewer comments and scoring variances in Round 1 and Round 2 don’t always inspire confidence, to put it gently.
It would be easy to say the problem was people at the Department of Education and a different team would have avoided these problems. I don’t think that’s the case. Rather, the competitions are thoughtfully designed but constrained by a flawed process (and of course a tortuous ‘gotcha’ politics).
That’s why Secretary Duncan must move quickly to head these problems off at the pass. The best way for him to do so would be to convene a commission or Secretary’s technical working group to study and report on what can be learned from the federal competitions so far and, more importantly, what can be learned from other high-stakes competitions in the public and non-governmental sectors. There are other fields, for instance, where a small subset of experts have to at once make decisions and manage conflicts of interest. What are best practices there? What are best practices for ensuring reliability among and between reviewers? What aspects of current federal grant making policy (which was really not designed for high-stakes competitions like these) should be changed? Is more training needed, and if so what kind? What else has to change if substantial amounts of federal aid are to be allocated this way?
Granted (ha ha) commissions are generally considered the place to go when you don’t want much to happen or want to punt an issue. But Secretary Duncan has proven clever at leveraging issues in creative ways and just by putting his brand on it and sanctioning a candid review and study he could make such a process meaningful. Substantive benefits aside, such a project would also help the Secretary make the case for moving in this direction.
Bottom line: The fallout here extends beyond states like CO, IL, and LA and it behooves the Secretary to get in front of it. Not in a way that invites pointless recriminations, but rather in a forward looking way that improves future initiatives.
Standing Disc: Bellwether personnel, including me, were involved in advising a number of states, winners and losers, about policy and strategy but had no direct interest in a specific outcome for the states mentioned here.