SIGnificant? Evaluating The School Improvement Grant Statistics

School turnarounds burst into the news last week when the Department of Education released some preliminary “snapshot” data on the School Improvement Grants or “SIG” program. Punchline: Surprise, mixed results!

Skeptics immediately pounced. My colleague Andy Smarick, a former state official himself and longtime turnaround critic, was all over the Department right out of the gate. He’s right, the results are not a ringing endorsement (see for yourself here pdf)) and the Department didn’t do themselves any favors releasing the data the way they did (on the day and in the way you dump bad news). Andy favors more dramatic interventions in school districts, in some cases I do as well, but while the record of new school creation compares favorably to the record for turnaround initiatives it’s far from a surefire fix. On charter schools in particular, plenty of places have managed to make a mess out of chartering. Just because districts are hapless in some places it’s not axiomatic that other solutions will be better.

So we should stipulate that fixing low-performing schools is enormously hard work and reasonable people can consequently look at the preliminary results the department released and see a cup half full or one half empty. There is also other emerging data on the SIG program, Stanford’s Thomas Dee found the natural experiment embedded in the design of the program and exploited that for a study. And Education Sector, Center for Education Policy and others have been tracking the where and how of SIG spending.

Unfortunately our political debate tends to lurch from one “finding” to the next in isolation. And it tends to be defined by the extremes of “this can’t work” to the opposite certainty of “we know what to do here.” The reality probably lies, as it often does, between those poles.  Turnarounds have a spotty record, it’s hard to do, most of the politically acceptable strategies don’t work, but it can work and we need bolder supply side strategies to in order to create new higher performing schools.  A mouthful? Sure, but this a complicated social policy problem we’re talking about here.

But even more than context, we should be asking which of the four basic approaches under SIG led to the biggest gains or biggest lack of gains, especially if any one approach did disproportionately.  Dees data points to more ambitious turnaround strategies. States did different things with this money and it supported different interventions. SIG is not a singular intervention, it’s funding stream. Analytically, that’s where the action is.

It’s also where the reaction to SIG has eerie (and unflattering) echoes of Reading First. A quick rush to judgment about a funding stream. In the case of Reading First it cost students – who needed support the most – a $1 billion reading program. Let’s hope history doesn’t repeat itself here, where the right and the left has reasons to dislike turnarounds. The real answers about what “works” or doesn’t and the lessons lie underneath the overall statistics released last week.

At Bellwether we work with people doing turnaround work, in particular the school turnaround program at the University of Virginia. And we’ve helped recruit talent to some of the most leading-edge turnaround efforts. The results of even the best turnaround initiatives are mixed, too, but caution against throwing the entire idea of turnarounds out.

For my part, I’m increasingly convinced that we’re thinking about turnarounds the wrong way. Many of the interventions states and cities are trying are weak – and they’re trying them on persistently low-performing schools. It seems to me the richest vein for lighter-touch turnaround work, which the field knows how to do, is mediocre schools, not persistently failing ones. In those places you have a core of good teachers to build around, solid support and training can make a difference, and a new leader can lead change.

The lowest performing schools need something more dramatic. That’s where the hardest work is, both substantively and politically. It’s where a strategy of providing new schools can make a big difference, too. But as a field we seem to have trouble talking about turnarounds with that sort of granularity. Probably because no one wants to acknowledge that the other “side” is at least partially right and there are no easy answers here.

9 Replies to “SIGnificant? Evaluating The School Improvement Grant Statistics”

  1. Dee is actually very careful to point out–if you talk with him about the paper–that his methodology does not allow him to say which turnaround model works best.

  2. Thanks a lot for sharing this informative blog with us! I am sure the school administration will pay attention to every requirement which a school needs.

  3. I firmly believe that schools can be turned around. However, if what states and cities are trying is weak, then even in “mediocre” schools the efforts are well, mediocre.

    I think the focus is less on the type of school and more on what it is we are attempting to turn around. Student performance will be affected only marginally by efforts that land on an existing industrial age system. We need to turn around the culture of public education with a methodology that changes attitudes and beliefs. By learning new methods for responding to student need, methods that represent a new paradigm, educators will adopt new ways of thinking about the job of education.

  4. For those with $1500 to spend (the teacher who wins the next PowerBall???)
    you can attend a delightful event up in New York:

    From: Mara Kane [capitalroundtable@…]
    Sent: Friday, November 23, 2012 6:22 AM
    Subject: For-Profit Education PE Conference — Harold Levy to Chair

    The Capital Roundtable is pleased to announce our ENCORE conference on —

    Private Equity Investing in
    For-Profit Education Companies —

    How Breakdowns in Traditional Models &
    Applications of New Technologies Are Driving Change

    Tuesday, January 15, New York City

    Sponsored by —

    • Drinker Biddle & Reath LLP
    • Parthenon Group LLC

    Private equity investing in for-profit education is soaring, and for good reason — the public and non-profit models are profoundly broken.

    This is why for-profit education is one of the largest U.S. investment markets, currently topping $1.3 trillion in value.

    You’ll take away a wealth of knowledge from this conference —

    Serving as chair of this ENCORE conference we are happy to welcome Harold Levy, former chancellor of the New York City School System and now managing director at Palm Ventures in Greenwich, Connecticut. Palm is a family office that invests in for-profit education and other capital investments with a positive and transformative effect on society.

    Harold leads the education practice, capitalizing on his vast experience in education and finance. He is also former executive vice president and general counsel of Kaplan Inc., director of Global Compliance of Citigroup, and head of litigation at Salomon Brothers.

    He has served on the boards of a number of academic institutions and for-profit education companies. Harold is presently treasurer of the Roosevelt Institute and a member of the Presidential Advisory Committee of Teachers College, Columbia University. He holds a B.S. and J.D. from Cornell University and a M.A. (PPE) from Oxford University.

    Register now — to learn how to achieve investment success in the education revolution!

    Along with Harold, our four panel discussions will feature the shared insight of 20 of the top private equity experts in the for-profit education marketplace, including GPs, LPs, operating executives, investment bankers, lenders, consultants, and more. They will offer real-world perspectives, lessons learned and industry outlooks, plus insights on managing current portfolio companies.

    Plus we’ve brought exceptional content and excellent networking together. We’ve built ample time into the agenda, including session breaks and an open lunch, when you can meet fellow attendees and featured speakers to exchange ideas and business cards.

    Click here to reserve your seat!

    For more information, click here. To contact me directly, feel free to call me at 212-832-7300 ext. 0, email me at mkane@….

    We look forward to hearing from you!


    Mara Kane
    Assistant Producer, The Capital Roundtable

    P.S. The Capital Roundtable is grateful to Drinker Biddle & Reath LLP and to Parthenon Group LLC for sponsoring this outstanding event. It’s due in great part to their collaboration that we can promise that this conference is going to be so worthwhile for you.

    P. P. S. Bring a colleague or client along, and receive a special discount. Please call me at 212-832-7300 ext. 0, or send an email to mkane@….

  5. Wow, and to boot, the location is a “Private club in midtown Manhattan.” Thanks Phillip, but I have to go be sick now.

  6. “Private equity firm Apollo Global Management LLC will buy McGraw-Hill’s educational publishing unit for $2.5 billion, McGraw-Hill said on Monday.”

    “Private equity firms have bought textbook publishers before. In 2007, Cengage, the No. 2 U.S. college textbook publisher, was acquired by Apax Partners LLP and OMERS Capital Partners from Thomson Corp, now known as Thomson Reuters, for about $7.75 billion in cash.

    McGraw-Hill received financial advice from Evercore Partners and Goldman, Sachs & Co., and legal advice from Wachtell, Lipton, Rosen & Katz and Clifford Chance.

    Apollo’s financial advisers included Credit Suisse, UBS Investment Bank and BMO Financial Group. It received deal financing from Credit Suisse, Morgan Stanley, Jefferies, UBS Investment Bank, Nomura and BMO.

    Apollo received legal advice from Paul, Weiss, Rifkind, Wharton & Garrison LLP and Morgan, Lewis and Bockius LLP.”

    I wonder if the brain trust at Bellwether considers such financial transactions a bellwether.

  7. Many of us who work in the school turnaround environment could have predicted the year 1 results USED released, some schools improved, some got worse. Unfortunately, that is to be expected in the first year of a turnaround effort. When modifying an existing system (with either tinkering or dramatic change), one still must fight against the status quo. There is no perfect model for a school turnaround and course corrections must be made along the way. As a result, the first programs and practices implemented might not have been the most effective and will be removed or adapted throughout the three-year implementation period.

    Honestly, I’m surprised that so many schools were able to achieve double digit gains in year 1. Year 1 schools often focus on culture and climate. While some academic improvements are likely, the real academic growth won’t come until instruction is the focus, in years 2 and 3. Just looking at year 1’s assessment data should not imply success or failure of a turnaround. We must also look at culture and climate indicators (i.e. student, teacher and parent satisfaction surveys, student AND teacher attendance, in/out of school suspensions, etc), in addition to student academic performance. If we’re evaluating turnaround efforts on assessment data for year 1, we are setting those schools up for continued failure. The schools (teachers and leaders) must know that they have the public’s support to implement a multi-year plan to make real and lasting changes. To say that a turnaround has failed so quickly implies that all of the improvements that were made were not effective, and in most cases this is simply not true, they just have a long way to go.

    The other major piece that’s missing in much of this discussion (i.e. all the blogs and the early research) is what changes are actually taking place in these schools? How does the achievement (and behavioral) data compare when you look at turnarounds vs transformation vs restarts? How does the political will of leaders (school boards, superintendents, principals) impact the turnaround effort? If teachers were replaced, was there a sufficient (and highly effective) pool of teachers to rehire from? Were community partnerships formed to address all of the social needs turnaround schools often have? How hands-on or hands-off was the state education agency in helping schools and districts make real and lasting changes? What systems, conditions, and practices were embedded into the larger district to ensure sustainability and continued growth (post-SIG funds).

    Until we look at 1) a more robust (and diverse) collection of data to get a real picture of what’s happening in a school over multiple years, and 2) the structures, supports, and implementation processes of turnarounds, it is far too early to draw definitive conclusions in support of or against the revised federal SIG program, and why it is or is not working. Until then, we must support the incredibly hard work that is being done in these schools, continue to work towards equitable educational opportunities for all students, and wait for additional research.

  8. There are a whole host of problems with the research about what is working or not on the turnaround front. Not only do we not know exactly what interventions worked in what settings but we really do not know anything about whether the changes will be sustainable when these large grants end. I too was a little surprised by how many schools had double digit gains. I find it unbelievable that there are not more large scale research studies looking at current turnaround work given such a large investment along with the importance of this work.

Leave a Reply

Your email address will not be published.