Why Is NAEP Flat Or Falling? Part 1

Last week’s NAEP results were received with less handwringing than you might have expected given that in the past relatively insignificant changes garnered a lot of attention. But the results matter – a lot. Especially if you consider it’s a precursor to the learning loss subsequently caused by the pandemic.

What’s going on? There are multiple theories and I’ve asked a few folks to discuss them here. Today Sandy Kress, education advisor to President George W. Bush and a former school board leader discusses accountability. Tomorrow school finance expert Marguerite Roza discusses money.  And we’ll also look at Common Core and the shifts in instruction associated with that reform. If you have another theory reach out and let me know.

Here’s Sandy’s take:

Accountability Works, Until It’s No Longer Accountability – By Sandy Kress

Although folks involved in K-12 policy disagree on many things, they will largely agree that the declines in achievement since 2012 are extremely disappointing and very worrisome. To learn of this lost ground after the economically strong decade of the 2010s and while we await even worse results coming out of the COVID period – this is all incredibly dispiriting.

Let’s look at samples of these discouraging outcomes.

In math, for 13-year-old blacks, the Long-Term Trend NAEP shows that while they made a nice gain of 13 scale score points (251-264) from 1999 to 2012, they lost a good part of it by 2020, back down to 256.

The bottom 10% of students fell from 240 to 228 from 2012 to 2020. The bottom 25% fell from 263 to 255.

In reading, 9-year-old blacks made impressive gains from 1999-2012 of 20 scale score points (186-206). Over the next 8 years, they were stagnant, falling back 1.

The bottom 10% of 4th graders grew 12 points from 1998-2009 but fell back 7 from 2009-2020. The bottom 25% grew 8 in the earlier period but fell back 2 in the latter.

So much for every student succeeding. We made progress in achievement in some states just before No Child Left Behind was passed and considerable progress nationally afterwards. But then we lost ground badly.

So, why did we make progress in the 2000s and go stagnant-to-bad in the 2010s?

Let’s begin with the data scientists’ warning: we don’t know for sure. Until we do serious cause and effect research, we can’t prove our assertions.

But we’d be irresponsible to leave it there. Policymakers and citizens can’t look at these trends and walk away, frozen from acting, because we can’t know for sure the scientifically proven explanation.

What we must do is put forward our best hypotheses and act on what seems truest.

Let’s look at some of the most popular explanations.

Could it be Common Core standards? Without citing the research, I’ll summarize it: Common Core had no significant impact, one way or the other, on student achievement.

Some say the drops may be due to the Great Recession of 2007-2009. Unless students were so severely traumatized, and on a virtually unprecedented basis, there’s no proof student achievement should have continued to fall for a decade after the recession ended.

Money? Some argue money makes a great difference; some say not. In any event, education spending increased for most of the years achievement dropped.

Instead, I want to argue that once we reduced or eliminated consequences for failing to make progress for disadvantaged students, we should not be surprised that their achievement went south.

It began when Secretary Duncan let consequences for failing to make progress be diminished by states that agreed to buy into the Administration’s teacher effectiveness and Common Core initiatives.

The weakening of the commitment to accountability was formalized in the passage of the ironically misnamed Every Student Succeeds Act. Then, states, in effect, could stay soft, and studies showed that’s what many did.

The solid research on accountability (Carnoy and Loeb, 2002; Hanushek and Raymond, 2005; and others) shows that accountability moves the achievement needle positively. Plus, we’ve learned much more over the years. If we use this knowledge and hold ourselves accountable for improving student achievement, it improves.

Improve by how much? It all depends. If our commitment is widespread and we fix problems that arise, improvement might be greater, perhaps far greater, than in the past.

Here’s the burning question for us now: dare we allow 2009 to be the peak of student achievement in American history?

5 Replies to “Why Is NAEP Flat Or Falling? Part 1”

  1. We have been doing the same thing over and over for decades expecting different results.
    Many children shut down/ disengage from learning to read due to confusion as a result of wrong teaching. They are instructional casualties (a word coined by Dr. Reid Lyon).
    All my more than 70 so-called dyslexic students whom I taught on a one-on-one basis since 2004, could read in Malay but not in English. I wanted to know the reason and quit my work in the audit field to research this.
    Let us discuss this.

  2. No, it’s not a lack of accountability that has impeded educational progress. I’d say that accountability–or at least accountability as we have known it, in the form of standardized test scores–has contributed to our LACK of progress. And it’s not just our failure to teach many kids to decode, as Michel Luqman has commented, although that’s part of it.

    Standardized reading comprehension tests, including the NAEP and the far more influential state tests, purport to assess general comprehension skills, like “making inferences.” As the recent NAEP Oral Reading Fluency study found (https://nces.ed.gov/nationsreportcard/subject/studies/pdf/2021026_2018_orf_highlights.pdf), the lowest-scoring readers (at 4th grade) are distinguished by their lack decoding and fluency skills. But we rarely measure the impact of those separately, and we basically never do it at higher grade levels.

    So we actually don’t have any idea how many low-scoring students can’t decode fluently, but it’s presumably in the hundreds of thousands, if not millions. The NAEP ORF study estimated that 420,000 4th graders fell into the very lowest category of “Below Basic,” which was distinguished by a serious lack of foundational skills–and that’s just 4th-graders. There’s no reason to think this problem gets addressed adequately at upper grade levels. So one part of the problem is our failure to teach many kids to decode.

    But the other HUGE part of the problem is that educators have long assumed that in order to boost reading test scores, they need to practice decontenxtualized comprehension skills, and it doesn’t really matter whether they’re acquiring academic knowledge and vocabulary. That approach, which is deeply entrenched and which was exacerbated by NCLB and–unintentionally–by the Common Core, flies in the face of what cognitive science has found about how reading comprehension actually works. For more on that, see https://www.theatlantic.com/education/archive/2018/04/-american-students-reading/557915/ and https://nataliewexler.substack.com/p/scores-for-low-performing-students.

    So yes, we need accountability, but not accountability that misleads educators and the public about what the real problem is. We should be testing decoding and fluency at all grade levels, and we should stop trying to test reading comprehension in the abstract — or at least, stop using those scores to hold teachers and schools accountable. Far better to just use, e.g., the NAEP history, civics, and science scores as a measure of general progress rather than the reading test, and at the state level to ground reading tests in the content of the curriculum, as Louisiana is currently experimenting with (if there’s no common state-level curriculum, which is the typical scenario, then at least ground them in state content standards).

    Sorry to go on so long, but you asked anyone with another theory to let you know, and I took you at your word!

  3. As to Natalie Wexler’s comments, I totally agree on the importance of teaching decoding and fluency. Indeed the Reading First component of NCLB promoted all the elements recommended by the Reading Panel and reading science.

    We saw nice gains at least at the 4th grade and 9 year old levels on the NAEP among disadvantaged students during the 2000s. But then Reading First was dismantled.

    Perhaps its dismantling ought to be studied as a cause of decline, too.

  4. One more point I’d make in response to Natalie: the two states that taught best to reading science AND were strong on accountability made the best gains on NAEP in reading – first, Florida, and, more recently, Mississippi.

    I surmise that effective teaching to what research endorses – along with accountability to impose consequences – are the right pair.

Comments are closed.