May 29, 2019

“What Works” in Education Is Not Merely A Question of Effect Sizes

Here’s a pet peeve: A champion of some particular education intervention will point to some research study showing Intervention X led to positive outcomes for participating students. Ok, great. Assuming it’s a good study, we now have evidence that Intervention X “worked” in a given Situation Y.

That does not mean Intervention X will easily replicate to new Situation Z.

Anytime you hear someone attach the phrase “high-quality” in front of some intervention, they’re talking about this problem.  These caveats pop up frequently in debates over particular reforms:

  • The small school reform effort produced long-term gains for students, but their backers largely abandoned them.
  • School integration efforts produced large gains for black students (with no harm to white students), but formal integration programs were and remain relatively small.
  • “High-performing” charter schools produce large gains for students, but there is wide variation in those results.
  • Teacher evaluation reforms produced gains in some cities, but the effects were smaller or non-existent when similar reforms were spread more broadly.

Some of these reflect implementation challenges. Others are more about politics (which is itself a particular type of implementation challenge). I am by no means the first person to make this point, but we can’t just say something “works” or “doesn’t work” without giving some consideration to where the policy worked, for whom it worked, what outcomes changed, and by how much it changed the status quo.

–Guest post by Chad Aldeman

May 24, 2019

Weekend Edu-Reads

“…any movement serious about improving education for low-income, rural, and minority students has to look outside of cities — especially in the South, where a majority of students live outside of city centers.” That’s Kelly Robson about the need for philanthropies to invest beyond urban areas.

“…the children of Perry Preschool participants — most of whom are now in their mid-20s — were less likely to be suspended from school, more likely to complete high school, and more likely to be employed full-time with some college experience. Children of participants were also more likely to be employed and to not be involved with the criminal justice system.” Read Marnie Kaplan on the latest research on the Perry Preschool project, which suggests the program had inter-generational effects.

California has been requiring prospective teachers to take a reading test with “no evidence that it contributes to more effective instruction.” Oh, and this same test is disproportionately keeping out black and Hispanic teachers. I suppose it’s good the state is considering dropping it now, but why did California start using this test in the first place?

“There is usually more variation in earnings results between programs within colleges than between colleges.” That’s Kevin Carey on what we can learn and do with program-level outcome data.

David Leonhardt and Sunil Choy partnered with the Urban Institute on this cool data visualization project on college dropouts.

The Pension Pac-Man must be fed.

–Guest post by Chad Aldeman

Posted on May 24, 2019 @ 11:01am

May 22, 2019

Absolute Versus Relative School Segregation

“Everyone knows” school segregation is getting worse, right? Well, no. That narrative has been fueled by a partial misread of the data.

It is true that absolute measures of school segregation are getting worse, but that’s mainly due to our diversifying country. In relative terms–how racially isolated are our schools compared to the underlying student population–segregation has not been nearly as dramatic. Here’s Brian Kisida and Olivia Piontek in a new piece over at Education Next:

In contrast, relative measures of segregation take into account the underlying composition of students, making them more comparable across locations and over time. They are also conceptually different in that they measure how evenly a given population of students is distributed across an entire school system. This makes intuitive sense, as segregation implies that some students are segregated from other students—relative to some underlying pool of students a school could enroll.

This issue with measuring segregation is well-known in the academic community, and there is ample scholarly evidence using relative measures of segregation that adjust for the underlying composition of students in school systems. Using these more sophisticated relative measures, such as the dissimilarity index and the variance-ratio index, examinations of trends find that segregation has been flat or modestly decreased over the past 20 years. In summary, massive resegregation is not occurring, and students are roughly just as evenly distributed across school systems as they were 20 years ago.

To be clear, segregation and racial isolation are serious problems in America and far more common than they should be, and efforts to shed a light on these problems are commendable. At every point in time between the Brown decision in 1954 and 2019, millions of American children have been educated in separate and unequal schools. This is an outrage that demands serious attention and action. But as we devise strategies and search for solutions, it is imperative that we are motivated by a complete picture of the problem we are trying to address. Segregation is a serious enough problem that it shouldn’t need to be worsening to be alarming. It’s bad enough as is.

–Guest post by Chad Aldeman

Posted on May 22, 2019 @ 11:52am

Education in the American South

My Bellwether colleagues Kelly Robson, Jennifer O’Neal Schiess, and Justin Trinidad have a new report out today on education in the American South. It’s a (long-ish) deck that provides a detailed analysis of academic outcomes in Southern states, placing them in historical, economic, and political context.

Read it here.

–Guest post by Chad Aldeman

May 17, 2019

Weekend Reading List

Fun graphs on the change in educational attainment over time by state.

Teach for America keeps getting better.

Hailly Korman and Kelly Robson on the fragmentation between schools and social service agencies.

Max Marchitello on how Arizona keeps cutting teacher retirement benefits.

Bellwether has two new short briefs on charter schools in California, on special education and facilities. In particular, the special education brief addressed one common myth: That California charter schools enroll fewer special education students and thus force traditional school districts to carry a disproportionate burden for educating those students. As the brief outlines, California uses a regional, census-based model that shares special education costs across entities and helps address this problem:

This structural arrangement for special education upends the commonly held belief that charter schools, through a kind of selective enrollment, choose not to enroll their share of students with disabilities, and thereby overburden traditional school district budgets. Instead, districts retain special education funding that is based on enrollment across both district and charter schools and determine how to best deploy those resources.

That funding arrangement system is still far from perfect, but it’s not as simple (or as bad) as the raw statistics might lead you to believe.

Today marks the 65th anniversary of Brown v. Board of Education. If you haven’t already, I recommend listening to Malcolm Gladwell’s podcast episode on the specific direction and argument the Court took in deciding the case, as well as the subsequent impact on black teachers.

–Guest post by Chad Aldeman

May 13, 2019

Money > Complicated Loan Forgiveness Programs

California Governor Gavin Newsom is proposing to revive a teacher loan forgiveness program. Here’s a brief description of the proposal via EdSource:

The college loan repayment program would provide up to $20,000 to an estimated 4,500 newly credentialed teachers who commit to teaching at least four years in high-demand fields in districts having the hardest time hiring credentialed teachers. Newsom suggested the program would make inroads in reducing the 6,000 to 8,000 emergency, uncredentialed teachers who have been hired annually to fill California classrooms.

Spot how many rules are buried into this paragraph? Candidates must:

  • Be newly credentialed teachers
  • Have loans
  • Teach in high-demand subjects
  • Teach in districts struggling to staff their schools
  • Verify said teaching for four years

Prior versions of this program required four consecutive years of teaching, and the loan forgiveness was all-or-nothing.

There are simpler, fairer ways to address teacher shortage areas. For example, a Georgia program offering supplemental pay for math and science teachers had large positive effects on retention. When Florida offered a combination of loan forgiveness and direct bonuses, both components seemed to have an effect on retention, but the direct bonuses were more cost-effective.

Instead of a convoluted loan forgiveness program, California should just send the money directly to teachers. Money should be paid out annually, not as some sort of promise if teachers complete all the requisite paperwork. Not only would this be more fair and simple for teachers, it also might just work better.

–Guest post by Chad Aldeman

May 9, 2019

The Side Effects of Reading Robert Pondiscio. Plus, What’s Eating Teacher Salaries in Kentucky?

Warning: Reading Robert Pondiscio may cause side effects of unintentional chortles and knowing grimaces.

Neerav Kingsland on New Orleans. His piece is a nice reminder against comparing effect sizes in isolation. It’s much easier to see positive effects of an intervention when dealing with a small sample, but getting the same effect size for an entire city, let alone an entire state, is quite a bit more meaningful in the real world.

Cory Curl with an excellent analysis of school spending trends in Kentucky. Like in California, health care costs seem to be a primary culprit for stagnant teacher salaries.

Lake Wobegon, New Mexico?

–Guest post by Chad Aldeman

May 8, 2019

Questions and Answers About Teacher Pension Plans, Boston Charter Schools, and Guns in Schools

Can we fix teacher pension plans? In a new piece for Education Next, I show it is possible for pension plans to offer adequate benefits to all workers, but it would require states to re-examine their current offerings:

There are two basic ways states could accomplish this. One would be to significantly increase the generosity of current plans. But to reach our adequacy thresholds this way, states would have to increase benefits substantially, essentially doubling the cost of their current plans.

Alternatively, there are cost-neutral ways for defined benefit plans to provide adequate benefits to all workers, but states would have to radically restructure their current plans.

Check out the full piece for more detail.

Can Boston’s charter sector scale up effectively? Apparently yes. The authors note that beginning teachers in Boston charter schools don’t suffer through the same steep learning curve that teachers in Boston’s traditional schools do, and they find that “the charter sector reduces variation in teacher effectiveness within schools, which may be due to charters’ centralized management of teachers and standardized instructional practices.”

On Twitter, Cory Koedel asked what would happen to school district insurance premiums if they started allowing employees to carry guns into schools. Here’s a good overview from Milliman, and one telling anecdote from Kansas:

At least 24 states across the country have policies that allow security personnel to carry weapons in schools, and at least nine states have policies that allow other school employees to do the same. But how could these laws affect school districts’ insurance policies and coverage? It’s not quite so easy, as the state of Kansas found out when it passed a law in 2013 allowing school staff to carry guns, and an insurer that covers most districts in the state subsequently issued a letter denying coverage to schools that took on this risk. Five years later, no Kansas school employee has carried a gun into a K-12 school.

Basically, insurance companies may play a quiet but important role in this policy question. Insurance companies are having a similar effect on school district and university decisions about whether they can afford to field football teams.

–Guest post by Chad Aldeman

May 7, 2019

Eduwonk Is Fishing… Plus Teacher Turnover, Diversity, Benefits, TEACH Grants, and the Peter Principle

Andy Rotherham has gone fishin’ for the summer, but in the meantime he’s given me the keys to the blog. I can’t match his wit or his knowledge of fishing, but I’ll try to keep it lively around here. Here’s what I’ve been reading lately:

New JOLTS data from the Bureau of Labor Statistics show, once again, that public education has some of the lowest turnover rates of any sector in our economy.

Kudos to NPR for sticking with this story on TEACH Grants.

Max Marchitello and Dr. Liso Grillo on how Howard University identifies diverse teacher candidates.

Cass Sunstein argues high school students applying to college are subject to excessive or unnecessary “sludge.”

Paul Bruno with data on health care costs in California schools. Short version: They’re rising much faster than other types of education spending, especially teacher salaries. Read the full report here or EdSource op-ed here.

Speaking of benefits, PDK published a good piece recently from James Shuls, Colin Hitt, and Robert Costrell on how teacher pension plans can exacerbate school finance inequities.

As we head into campaign season, Conor Williams asks what’s the best way to spend billions of dollars of new money to improve outcomes for low-income students. Is it teacher salary increases, or something else?

Alex Tabarrok on a new study on the “Peter Principle,” the idea that people keep getting promoted up the ladder until they’re no longer good at their job. There are implications here for the education field, particularly in how to think about keeping great teachers in the classroom.

Mocktail bars?

–Guest post by Chad Aldeman

Gone Fishin’…But You Are In Good Hands With Chad Aldeman

I’m taking some time off this year, an extended amount to focus on some non-work things. Basically, with the exception of a few commitments I’m out through October. I’ll be traveling, riding my bike a lot, hopefully bending fly rods, and learning some new things unrelated to my day job. I’m looking forward to a chance to recharge and stretch my legs, figuratively and literally. What I won’t be doing is working much, and that includes blogging.

As I noted in the most recent Bellwether newsletter, a funny thing about the professional world is that when you tell people you are taking time off, they immediately assume something must be wrong or at least going sideways. That’s not the case here — instead, something is right so I can do this. While I’m gone Sara Mead will lead our policy and evaluation work. I have known Sara for almost two decades: she’s as good as they come, and the team will thrive on her watch. I’m deeply grateful to the team at Bellwether that we have an organization that can accommodate life things like this.

But what about Eduwonk? Things around here will be in the capable hands of education polymath Chad Aldeman, who I’ve also known and worked with for a long time and who, like Sara, is a gem. Burning questions include, ‘But will he do fish porn?” That, like all decisions, will be up to him. When I turn the blog over to guest bloggers, and over the years there have been dozens of them from all factions of the ed world, I don’t backseat drive. It’s Chad’s show, he’s good, and so no matter what he does in terms of style or content, I think you will like it.

See you in the fall!