In this week’s TIME column I take a look at PineappleGate. In case you’ve been under a rock that’s the saga in New York over a question on the state’s ELA test. I trace the saga from the incorrect passage first published by the NY Daily News to the larger implications for our conversation about testing. The column includes an exclusive comment from Pearson – a deviation from their usual policy of not commenting on these issues – as well as a leaked memo outlining the background and use of the question.
When the New York Daily News posted an article about an Aesop-inspired fable that appeared on the standardized test eighth graders in New York state had to take last month — about a pineapple challenging a hare to a foot race through the forest — all hell broke loose because the passage was so poorly written and the questions about it so incomprehensible. The fable described several animals assuming that the pineapple must have a trick up its sleeve that would enable the immobile fruit to win the race, and when they discovered that it didn’t, they ate it. Test-takers were asked: Why did they eat the pineapple? The correct answer: because the animals were annoyed. And who was the wisest of the animals? An owl that was never mentioned in the passage. Anti-testing activists responded with fury that this set of questions showed why standardized testing is worthless. New York officials quickly turned tail and tossed out the pineapple passage, declaring that they would not count it on this year’s test and would not use it in the future.
There was just one problem: much of the uproar was based on bad information…
Hmmm. This feels like an awfully defensive response. The author of the “pineapplegate” passage (not the bowdlerized version published by the Daily News) has been quotes expressing his suprise that his work could be used in standardized tests at all. “As far as I am able to ascertain from my own work, there isn’t necessarily a specifically assigned meaning in anything,” Daniel Pinkwater told the Wall Street Journal. “That really is why it’s hilarious on the face of it that anybody creating a test would use a passage of mine, because I’m an advocate of nonsense. I believe that things mean things but they don’t have assigned meanings.”
So this is not a simple case of an “uproar based on bad information.” I think what you’re missing in this fairly strident defense of standardized tests and testing is that we are suffering generally from a case of test fatigue. The entire edifice of reform rests on testing, and while the principle of accountability is sound, the effects of testing on schools has proven to be deleterious, to be charitable. Thus the conditions are ripe for people to overreact to perceived absurdity in the tests. And that’s exactly what happened here.
Seeing the “correct” version of the test passage and questions did very little to reverse the impression that this was a questionable measure of students’ ability. More competition or a single national test will not solve the problem of poor passage selection on tests. What might solve the problem is choosing passages based on the content that kids are actually taught — the adoption of something like a national scope and sequence with test items chosen exclusively from the list of topics and works of literature cited. To do otherwise reinforces the demonstrably wrong impression that reading comprehension is a “skill” that can be applied to any randomly chosen reading passage.
Hi
Thanks for the comment. We’d probably disagree on some of this but speaking of close reading the column makes the point you do about the need for better passages than this. It’s clearly suboptimal but that is not the same as indefensible. It is unfortunate, however, that in the din the issues raised in Ravitch’s 2003 Language Police book did not get more attention. Big issue here.
Yes, the answer is to dump standardized testing, as least when stakes are attached.
Reformers are still stumped by Supply and Demand 101. Bubble-in tests have always been lousy (for the purposes that they’ve been assigned for evaluating schools at scale). So, NCLB artificially stimulate demand, so of course the supply went up. In what universe would the quality also go up? For that to happen, you’d have to postulate that Pearson et al care more about kids and profits …
The same question still applies to advocates of better standardized tests. Econ 101 says it won’t happen, as the Pearsons of the world get richer.
And not to change the subject, but I have yet to hear a reformer describe a plausible scenario where value-added evaluations does not produce an outmigration of teaching talent from the inner city to schools where it is easier to raise test scores. Firing teachers will increase (rightly and wrongly) and it will increase demand in suburbs and in charters, magnets etc., as well as all other schools.
Since value-added will always be stacked against poor neighborhood schools where it is tougher to raise test scores, the demand for scapegoats will continue to soar. Inner city teachers will be subjected to even more pineapple-type mania. Make teaching in the inner city even more maddening and absurd and the supply of effective teachers willing to put up with it will collapse.
The supply of students who retain their love of learning in the faceof this testing madness seems to also be collapsing. At minimum, its wrecked the last quarter of their school year.
If it’s between suboptimal and indefensible, it’s probably a lot closer to the latter–especially if one of the cherished goals of the accountability movement, to close the achievement gap, is to be taken seriously. More here:
http://prospect.org/article/theres-no-such-thing-reading-test
rp
Robert makes very good points, including: ” I think what you’re missing in this fairly strident defense of standardized tests and testing is that we are suffering generally from a case of test fatigue. The entire edifice of reform rests on testing, and while the principle of accountability is sound, the effects of testing on schools has proven to be deleterious, to be charitable.”
John Thompson also makes very good points about the (perhaps unintended?) side-effects of all this testing mania on supply and demand for teachers in low-income schools: The ironic effect of these reforms is that individuals are incentivized to avoid teaching in the low-performing schools these measures were ostensibly intended to help.
It’s bizarre that so many people are embracing and promoting reforms that make teaching in low-income schools even less of an appealing prospect for potential teachers!
Good post, AR.
This messy situation should remind us all just how simplistic was the 1990s notion that test-and-verify would lead us, somehow, magically, to significant improvement in achievement.
Beware where one steps when running on a freshly tarred road.
btw: Linda D-H’s provocative piece here: http://www.washingtonpost.com/blogs/answer-sheet/post/education-and-the-income-gap-darling-hammond/2012/04/26/gIQAHn0LkT_blog.html
deserves serious attention.
As Mr. Rotherman correctly states in his Time article, this is a topic that requires critical thinking.
In the first place, when something “goes viral” it is usually because the topic hits a collective nerve. In the case of The Pineapple and the Hare, many citizens finally saw the idiocy behind all the standardized testing that is going on across the nation. Whether the Daily News got it exactly right or not is irrelevant because the story, as it is, represents the stupidity of the movement that we are in. I always knew that once the fog of the recession started to clear, people would begin to see what is happening to their schools and their children. That’s what’s happening now.
In order to comprehend this idiocy, one must first understand the purpose of a standardized test. Basically it is designed to measure a child’s level of academic achievement compared to other children of his age. So if a child scores at the 55th percentile in reading, it means that he reads better than 55% of children his age across the nation. THE ACCURACY OF THIS TEST RELIES HEAVILY ON ITS INTEGRITY. In other words, the test will only be valid and reliable if the test taker has not seen it before and no one tampers with it afterwards. A criterion referenced assessment is different but still depends on strict test security if it is to give accurate information.
When I was a child, the teacher would just hand out the test booklets one day and we’d all take it. Several weeks later our parents would get a report telling them how we did. There was no test-prep or high pressure beforehand and no one knew what was on them. The information the parents got was fairly reliable and would give them a general idea of how their child was achieving compared to other children of the same age.
With “high-stakes” all that has changed. Although the test publisher might be guarded about the content of the tests, once it gets to the school, there are many, many ways of finding out what is on them. In some cases, principals deliberately leave them out so teachers can “review the directions.” In other cases there is outright cheating, as all know from reading recent news reports.
A standardized test depends on “sampling” to find out if a child knows about a subject. So if a student knows 8×6 and 5×4 he probably knows his times tables. But if the teacher drills on those two items, the child might only know 8×6 and 5×4. If the teacher drills her students on all items, a child might score “proficient” when in reality he only knows the items on the test.
This test prep is occurring in mostly urban schools across the country, making it look as though “miraculous” improvement is occurring when we know from the NAEP that there has been steady but small improvement each year. Parents in places such as DC have complained that their fourth grade child scored “proficient” in math when the parent knows that the child can barely add.
These scores form the basis for firing teachers and for research done by university professors. Many people think it is a deliberate fraud designed to discredit our teachers and our schools.
If we truly want to measure the progress of students each year, the tests must become “low stakes” again. If citizens want to evaluate the work of the teacher, then there must be another professional who can monitor and evaluate the progress of her students. There is no ten-dollar group test that will do this important job. This is just common sense and I’ll bet everyone knows it too.
As for university researchers, I can’t imagine why they would base their research on scores reported by schools (“Oh, look at us. Our scores went from the thirteenth percentile in 2011 to the ninetieth this year!”)
Do you support standardized testing as it is administered to our children at this time? If so, you need to think critically and ask yourself why.
Great thread here. Everyone has something really critical and vital to say.
Questions: it often seems that urban schools are failing schools. Yes? If so, why is that? Why do we rarely talk about suburban schools which fail standardized tests? Are there more unionized teachers in urban schools–is that why they don’t do so well? Are people drawn to teach in urban schools failures themselves? Are urban districts just somehow of low quality and that is why they can’t compete with suburban school districts? Do suburban schools and districts and teachers know something or do something that urban schools don’t do because urban educators don’t know how to do/don’t want to/are too lazy to/ are too unionized to do?
Linda: You make excellent points (above). If only you were our Secretary of Education instead of Arne Duncan, what a better world this would be (seriously).
To response to Jeffrey Miller’s comment regarding why ‘urban’ schools do so poorly, the obvious answer is that the students who attend these schools are very different from suburban students, and the problems they bring to the schools are hard for the schools to deal with. In my experience, a ‘good’ school is a school attended by well-behaved, middle class kids with educated families. A ‘bad’ school is usually simply a school attended by kids with many societal disadvantages (these students also often have many social/cultural behaviors that work against school success).
The solution, in my mind, to these problems has little to do with the teachers in one schools vs the other.
Instead, we need to be honest about the problems in these schools, which often involve serious discipline and behavior problems on the parts of the students (including students threatening teachers, refusing to sit down or work, and refusing to do any school work or homework).
Pretending that it’s the fault of a teacher when a 16 year old inner city student doesn’t come to class, screws around when he does show up, and consequently does very poorly is simply burying our heads in the sand, AND is a surefire way to convince teachers and potential teachers to avoid teaching in these schools.
My apologies for the typo above — my first sentence was meant to start with “To respond to Jeffrey Miller’s comment”.
Thank you, Attorney DC. You have made my day! President Obama could do so much for education just by appointing a successful teacher or professor to the job of Secretary of Education.
There is some good news. Read “What Teachers Want” by Dana Goldstein in The Nation. There is some indication that Bill Gates is beginning to see some of the unintended consequences of his policies. Once he gets teachers involved in real reform, he’ll begin to see the improvements that he is striving for.
AttorneyDC makes excellent points in her response to Jeffrey Miller. I’d like to add to it with some personal experiences.
I have twin grandchildren, a boy and a girl, who are seven months old. They are the children of my son, a city councilman, and his wife, a college professor. I go over to their house twice a week to play with the babies.
No matter what the twins are doing, when I start to talk to them, recite nursery rhymes or read Pat the Bunny, they become instantly quiet and focus intently on what I’m saying. It’s a fascinating thing to watch because you can almost see their little brains soaking up language right before your eyes.
What is happening to the twins is called “informal education.” It happens at home, on the playground, at the neighbor’s house, at Grandma’s. But it does not happen to everyone. If you go to the house of another infant, you might find the baby propped up with a bottle, alone in his crib or staring at a TV screen. Many of their parents don’t know how critical these early months are to a child’s later education. Once I gave a book to the parent of a year-old baby. The mother laughed and said, “Linda, you know she can’t read.”
In the book “The Meaning Makers” we see how informal education influences the academic achievement of the child. My twin grandchildren, with years of books and talk, will likely be advanced when they enter kindergarten. The child without enriching experiences, will likely be very much behind. This is the “education gap” that we speak about and it’s very much established by the time the child gets to school.
The affluent, high-scoring school is often filled with children from literate homes. The low-scoring school often have many children who have not had these critical preschool experiences.
There is a mountain of research to back up what I have just said. This is the “data” that everyone seems to want. Now we must apply it and provide equal opportunities for our least advantaged children. These opportunities must begin when the child begins – at birth.
Linda: Good points, as always. Obviously, the child’s parents and home environment have a tremendous influence on his or her academic skills. I’m never sure why education reformers and other policy gurus hide their eyes and pretend it doesn’t. Perhaps it’s easier politically to blame the middle-class teachers than the low-income parents?
I am educated in math and physics. I spent my career on submarines. Now I work in a university physics lab.
I work hard. I think hard. I get paid a decent salary.
But I see gold in the hills. Mr. R, I want a job in edu-reform. I want to be the state spokesman/person/thing for teacher reform, accountability, testing, merit pay, accountability, talent recruitment coordinator and development recruiter, quality leader, and testing quality manager.
Can you help me out. I want a piece of this action. I want the pay, the anonymity, and the entire lack of accountability. I would do it for ten years until this fad passes or just until taxpayers catch on to my little cozy sinecure.
What can you do for me? Physics and math have too much accountability and the work is a bit much at times. And I want a BIG TITLE.
If you need a little humor on this high stakes testing subject, here is a video about the new STAR test (Pearson-created end of course standardized tests) that is making the rounds of Texas schools at the moment