More CMO Study! And The Study

Here’s the link to the study itself (pdf).  From yesterday’s TIME column here is the sum takeaway on performance:

Of the 40 CMOs that were selected for inclusion in the study for various reasons, including having a minimum of four member schools, 22 networks had sufficient data for the student-achievement analysis, which looked at three years of middle-school performance. The study found that, in general, students at charter-network schools outperform similar students at traditional public schools, although sometimes not by very much. But that overall average masks an enormous variation among different CMOs. High-performing CMOs are so effective they are providing the equivalent of three years of schooling for students every two years. But CMOs at the low end are so bad they are effectively costing students a year of learning every two years. Bottom line: 10 of the 22 CMOs are outperforming their public-school peers in math and reading, in some cases substantially; eight are middling; and four are serious laggards.

If you follow the issue closely don’t miss the – cliche alert – treasure trove of descriptive information in this study about how CMOs are operating. A few pro and con commenters have opined to the effect that this data must be either “forged by public schools and teacher unions” or not valid because it doesn’t use a pure RCT or randomized model as, for instance, Caroline Hoxby does in her research on charter school effects.  The methods are solid, learn about them yourself at the link above.  The focus on middle schools stems from a data availability issue.  It’s harder to do longitudinal studies for elementary school students because most states don’t assess in the early grades and at the high school level the assessment policies are very mixed, creating data issues there and necessitating the use of other measures – eg graduation, college-going, etc…in addition to this ongoing research effort the new Broad Prize for CMOs will also shed some light on those issues.

How you interpret the performance data probably has a lot to do with that you think about charters and CMOs in the first place. My take is two-fold.  One, given where a lot of CMOs operate I’m not surprised by the quality issues.  A problematic mix of poor quality charter authorizing and badly designed state policies create an environment where school replication is not always a function of quality.  But, while we can certainly do better there, even with improved laws and better authorizing no one should expect 100 percent success.  There is inherent risk in creating new entities.  Worth noting that even the very good CMOs have some individual schools that struggle.  In my view the question is how much risk are we willing to tolerate and given that we’re talking about schools, how much should we tolerate?

Update: Good overview from NSVF’s Jim Peyser.

8 Replies to “More CMO Study! And The Study”

  1. I always need to consult experts before speculating on the size of the impacts, but if this table is accurate, I’m not surprised. The report said,”Among CMOs, school-wide behavior policies and intensive coaching of new teachers are positively associated with student impacts in both math and reading.”

    Gosh, can’t we all agree to support those two practices?

    And the study also found, “At the CMO level, we do not find impacts to be associated with use of a uniform curriculum, extended instructional hours, frequent formative student assessment, or performance-based compensation.”

    Gosh, shouldn’t we all agree to reject the top down curriculum “reforms” that are sapping the soul of education? I’m not a fan or an opponent of the other three. The study should call into question “reformers'” near-religious beliefs in those ineffective practices.

  2. CMO= Cover My Orifices?

    MEanwhile, back in the real world where Andy does not venture:

    ST. LOUIS • When students first entered Imagine Academy of Academic Success four years ago, their school was already entangled in a complex series of real estate deals — ones that would divert dollars from their education.
    By the time they were on their first summer break, their brown brick building at 1409 East Linton Avenue had been sold three times, the final price nearly 10 times higher than the first. In the process, the company running the school — along with a small group of other players — cashed in.
    Imagine Schools Inc., the nation’s largest charter school operator, runs six charter schools in St. Louis. Together, their performance on state standardized exams is worse than any school district in Missouri.
    Nevertheless, those schools are generating millions of dollars for Imagine and a Kansas City-based real estate investment company through real estate arrangements ultimately supported with public education money.
    The deals are part of a strategy that has fueled Imagine’s national expansion. In most cases, Imagine sells its buildings to another company that leases them back to Imagine, with the schools themselves shouldering the rent with public funds.

    Read more: http://www.stltoday.com/news/local/education/imagine-schools-real-estate-deals-fuel-company-growth/article_dbf9b959-0c73-586c-97e7-6fca3a729b39.html#ixzz1cmZG2pW1

  3. John … The Mathematica study was a study of CMOs, not a broad study of curriculum and instruction, so the results provide no broadly generalizable evidence concerning practices that you believe should be adopted or those that you believe are “sapping the soul of education.”

  4. Really??!! Why is everyone surprised and fighting over this? Some CMO’s are good and really want to improve education for their students. The others… let’s just say that education as a business opportunity (edu-business) is the order of the day.

    The bottom line is that it’s really up to the board of he authorizing district to decide whether money or quality of service is the motivating force behind allowing crap CMO’s to continue or not.

    So you can say CMO’s are evil or CMO’s are great. But, the policy debate on this is really not at the level you think and with budgets being what they are….good luck.

  5. The study also found, “At the CMO level, we do not find impacts to be associated with use of a uniform curriculum, extended instructional hours, frequent formative student assessment, or performance-based compensation.”

  6. More on Imagine (and not the John Lennon kind):

    ST. LOUIS • The charter school operator Imagine Schools has placed a top executive on administrative leave and has received a stern warning from the sponsor of its St. Louis schools, following revelations about the company’s financial dealings.
    Top officials at Missouri Baptist University met this week with Dennis Bakke, chief executive of Imagine Schools, for a “candid and honest discussion” about the university’s relationship with the company, said university spokesman Bryce Chapman.
    Findings about Imagine’s St. Louis real estate and contractor dealings in recent days are ‘serious” and “need to be investigated.” Chapman said.
    Bakke has been in town this week giving Imagine’s six underperforming charter schools in the city extra attention and support after a Post-Dispatch series showed them to be entangled in complex real estate dealings that have contributed to high rent payments while classrooms lacked basics such as textbooks.
    The company has placed Sam Howard, executive vice president, on administrative leave pending the results of an internal investigation. Imagine spokeswoman Lori Waters said. The Post-Dispatch reported that a contracting firm that renovated Imagine school buildings had made $32,000 in payments to a “Sam Howard.” Howard had no explanation for the payments.
    The Missouri Public Charter School Association is now calling for an investigation of the schools by the state auditor.

    Read more: http://www.stltoday.com/news/local/metro/7bc516eb-db20-5a8e-984a-d4fa7c7edf53.html#ixzz1d4qiqWRj

  7. “a pure RCT or randomized model as, for instance, Caroline Hoxby does in her research on charter school effects.”

    Or for instance, the much larger national study of charters schools done by Mathematica.

    By the way. neither of these studies is an RCT. They are both lottery studies, which have a randomization device (lottery) but of course the lottery is non-binding so these should be considered natural experiments or good quasi-experiments if handled correctly.

  8. The bottom line is that it’s really up to the board of he authorizing district to decide whether money or quality of service is the motivating force behind allowing crap CMO’s to continue or not.

    So you can say CMO’s are evil or CMO’s are great. But, the policy debate on this is really not at the level you think and with budgets being what they are….good luck.

Leave a Reply

Your email address will not be published.