Grist For The Mill…

I can only imagine that this sensible Times overview story about test scores in New York will occasion yet another round of debate…(and by the way, this issue is one where there are actually unsettled issues, conflicting evidence, and real nuance).

15 thoughts on “Grist For The Mill…

  1. john thompson

    Yes, we need rel nuance. Yes, we read the conflicting evidence in the NYT, WP, USA Todat, and other newspapers and in scholarship from great universities, as was cited in the article.

    My favorite parts of the story were a couple of quotes that illustrates how adults look at unsetled issues. A 9 year old like standardized testing because “I like making my brain smarter.”

    On the other hand, Joel Klein demonstrated nuance by saying “No matter how you look at them,” he said, “the picture is one that shows that the city is making dramatic progress.”

    In contrast, this sensible overview cited Dr. Everson, a senior research fellow at City University of New York and chairman of the Technical Advisory Group, a collection of experts who oversee the state testing process, said he believed New York’s tests were “about as good as we can build them.” But he said that with so much riding on the tests, there was a need for greater study to certify that rising scores correspond with an increase in learning.

    “Unfortunately, I think some of the gains that we’re seeing are probably related to test-score inflation, meaning that people are using inappropriate ways of teaching to the test,” he said. “Instead of really good time spent on instruction, they’re doing a lot of test prep and drill.”

    Dr. Everson said that “the skepticism that you hear and I hear is real skepticism, and it’s warranted.” “Are things getting better?” he asked. “That’s what we really want to know, and we need more research to tell us how to sort that out.”

    An excellent example of the trade-offs came from a
    principal whosaid her own son suffered nerve-induced stomach aches several years ago when test-preparation began at his high-performing Queens school. But at P.S. 398, which had struggled for years with low scores and discipline problems, she has come to feel that the push to raise scores has brought genuine gains in knowledge.

    She acknowledged that there were casualties of the school’s efforts to raise its numbers — like art classes

    That brings up a serious question. Why concentrate so much on policies that have so many inherent trade-offs? NCLB has undoubted done good. It has equally undoubted done harm. And at its price tag, not to mention the pricetage of Kleins efforts, how could we have not produced something more cost effective?

    Preshool is a win win. Community schools are win win. Data-informed decision-making is a win win. Collaboration is win win. Improved nutrition and “green schools” and expanding PE, art, music, and enquiry is win win. Respecting social science is win win.

  2. Steve Peha

    When my clients ask me how to help them raise test scores, I am very honest with them: teach to the test and test your kids frequently. Nothing works better. (Nobody takes me up on the offer, which is good because I don’t offer the service.)

    But if someone wanted an interesting study, that would be it. How does teach-to-the-test instruction and frequent testing raise scores on standardized tests?

    Very simply and very effectively. Thank you very much!

    We all get used to things. And the things we get used to get easier over time. Slimming the curriculum down to two subjects (reading and math), teaching the exact types of problems that will appear on the test, testing your kids formally up to 8 times a year (as NYC does), and generally focusing all of your energy on raising scores will, indeed, raise them.

    But John Thompson asks, “at what cost?” And this is where the nuance comes in.

    Unfortunately, nobody wants to look seriously at this issue. No one that I know of is studying it or has studied it in a serious way. And no one is taking very seriously about the competing data that districts end up with on things like the NAEP and the SAT.

    We all want to believe, don’t we? We all want to think that there’s a solution out there that every school can implement. And even though we say we don’t believe in silver bullets, we sure do get excited when someone starts shooting.

    Unfortunately, the casualties in all this, as usual, are children. I’ve visited many teach-to-the-test schools and even their prize pupils can’t think their way out of their own paper lunch sack. If I ask them something that actually requires thinking or, heaven forbid, writing, they clam up and shut down. Without something to bubble in, they really struggle just to understand the form of a particular intellectual challenge — especially in the area of writing which requires the coordination of so many intellectual skills.

    Are the kids in NYC learning more than they used to? Probably. Are they learning enough? No way. Not even close. And I think the most nuanced truth here is that both Bloomberg and Klein are smart enough to see this, too. It’s just not in their interest to talk about it.

    At some point, perhaps many generations hence, we will all begin to realize that learning is not about bubbling in answers on an answer sheet and that standardized tests are not accurate measures of knowledge when it comes to individual students. For one thing, the margin of error is too high. And for another, as one person quoted in the NYC story said, they’re just too predictable.

    As we all know, familiarity breeds contempt. So it’s no surprise that the more we teach to the test and the more we test our kids, the more contemptible our schools become. Just as teachers rightly argue that they should not be judged by test scores alone, neither should kids. What’s at stake is nothing less than fairness and freedom of intellectual expression. The folks in NYC give up a lot to get their scores. I sure hope they’re happy with the results.

  3. Win-win

    Teachers on summer break who have too much time to post overly verbose comments every day on someone else’s blog going back to school in a few weeks is win-win.

  4. Chris Smyr

    I disagree with the assertion that testing breeds students who cannot think. I disagree with this because I find it illogical. Giving a student a question to answer that requires 1 correct answer out of 4 possible answers REQUIRES critical thinking and logic to answer correctly, even if it is not an open-ended assessment item.

    I can drill my students all day everyday on the vocabulary, sentence structure, and likely topics they will see on standardized tests, but if I fail to include elements of critical thinking into their curriculum they will not improve their test scores. They need to know how to mentally connect concepts before they can express their understanding on a test, any test, even standardized ones.

    There’s also a false dichotomy being steadily formed between the use of standardized tests and intellectual expression/challenge that needs to be addressed. That standardized tests focus on content knowledge related to content standards does not invalidate their ability to assess students’ understanding. There are not many who will say that content knowledge is the ONLY thing students ought to learn in school, but standardized tests give a quantitative way of assessing their understanding of such and comparing it to students on school, district, and state levels. Data used from this can be utilized by schools as they fine tune instructional strategies that lead to evidence of higher learning. All the while, teachers target different aspects of learning (motivation, language fluency, critical thinking) so that students are learning enough to show improvement compared to other classes, as all of these things come together to help students raise their test scores.

    I’d also like to meet this teacher that uses state test scores as the ONLY form of assessment in his or her classroom, as most teachers I know utilize alternative forms of assessment to keep their students motivated to achieve and focused on acquiring knowledge. Pardon me for being a non-believer.

  5. john thompson

    Win Win, Wondering, and Anonymous have a similar way of expressing themselves, starting with an attack on the commenter not the comments and then following the same logic and ending with a comparable punch line. Surely the readers of this blog are too old to have had their writing style designed by the standard test prep. Otherwise I’ve be afraid that education “reform” was already producing a generation of cloned readers and ways of expression.

    But seriously, readers and writers of this blog have probably had great educational opportunities. Its students like mine that are damaged by the unintended effects of NCLB-type accountability.

    Over at the NYT, a number of widely respected educational experts just wrote on somebody else’s blog the same thing about NYC scores that I wrote.

  6. john thompson

    Ok, I’ll take the bait. “Wondering,” I believe, asked about my value added. When I taught a tested subject I had a pass rate of 85% which was more than double the department average but another teacher had a higher API because he had more high paases. By that year’s rules, Limited Knowledge was called passing. So, by that years rules, I had a 100% pass rate. And back then whether it was CRTs of the more common NRTs my scores always were double the rest of my department.

    Now our school has a pass rate in my old subject of 25%.

    But honestly, I would quit the classroom before I taught another EOI tested class. I did work harder when all eyes were on my class and we were all working our tails off to turnaround the school. My students also worked harder and learned more of some things. And I think I did a good enough job of managing my students’ and my’ stress.

    I can get away with this because because everyone has seen me teach.

    But raising test scores wasn’t worth it. Life is too short. I’ve stayed in the inner city classroom for complex reasons – the adrenalin rush, the students, the learning etc. But my relationships with students trumps all. I can’t honestly say that students gained enough for the trade-off to be worthwhile.

    So, on the few occasions when I fess up to my past pass rate, I always emphsize the other side. I could never get that pass rate anymore. Back then, we were basically an inner ring suburban school. We had more violence back then. Today we have more students who have improved their decoding skills. But we have a greater critical mass of traumatized kids now that makes us a much much tougher challenge.

    Honestly, I couldn’t care less about test scores. I’ve got more important things on my mind and in my heart. Lastly, I tell my students, “I could give a flying ___ about test scores. I want them to be sitting in class next year knowing that they are prepared. In three years I want my students to be in a college class knowing that they may not know as many facts as the suburban kid on their right or the suburban kid on their left, but I want my students to have the earned confidence that they belong. I want them to KNOW that they have learned to learn and that they are ready. We’ve got three years to get there and I know what it takes, my former students who know what it takes will being returning to pass on their experiences. And I know that each person in class can do it. A lot of you all have brothers and sisters who’ve been in our classes and you should listen to them. I also add that I know I can’t get it done without being honest. Sometimes you won’t like my honest appraisal. But if we’re honest with each other we can get it done.”

    Honestly, I don’t know how I could serve two masters.

  7. Chris Smyr

    It is difficult to understand what you write sometimes, but to tease out some points to respond to:

    “But raising test scores wasn’t worth it. Life is too short.”

    Does that imply that your test scores are not linked to what your students were learning? If so, what were your students learning if it wasn’t content knowledge? And why did you choose not to focus on content knowledge or standards?

    “In three years I want my students to be in a college class knowing that they may not know as many facts as the suburban kid on their right or the suburban kid on their left, but I want my students to have the earned confidence that they belong.”

    That sounds akin to lowering expectations for your students (“You may not have learned what others learned, but be confident that you’re just as smart”). Do you think that’s a good path for other teachers to take?

    “I want them to KNOW that they have learned to learn and that they are ready.”

    How will you know that they have “learned to learn”? And what does that mean to you?

  8. john thompson

    Chris,

    If I haven’t learned enough over the years that my intuition is a whole lot more reliable than test scores, then something is wrong.

    This scores may or may not mean something. That’s life. With experience you learn professional judgment but nobody will ever know. That’s another great thing about the adventure of life and learning.

    No, I am not lowering expectations. I didn’t disparage content knowledge. Overemphasizing facts and skills is lowering expectations. Where the balance lies, that comes from intuition. But when I took into the opportunity costs of teaching some more content, it wasn’t worth it. You can’t make the bricks of concepts without the straw of content. But neither should you confuse the bricks that are the real goal with the supporting straw.

    From the time I was a little kid, a range of adults taught me that the key is “learning how to learn.” Do you really not understand what I’m saying? Aren’t kids today being taught that that the fundamental goal is learning how to learn? I really want to know the answer to that. You are worrying me.

    While I have you, let me ask this. Wouldn’t you agree that every time you teach a child by your behavior that there is a “right” answer you risk negative consequences? I’m not saying we have to play Hamlet all the time and open Power to the People schools, but surely you agree that learning through standardized tests has inherent harm? Maybe the good outweighs the harm. but surely you are aware that there is a downside?

  9. Chris Smyr

    That you think your intuition is more reliable than data is one of the real issues here. That’s what makes this conversation difficult, because there are teachers who think they know better than some “cold” standardized tests, even when those tests offer valuable data that can often better inform their instructional practices. Without evidence of student learning, you have nothing. Intuition can be biased and haphazard, even for veteran teachers, and its prevalence is one of the reasons that achievement gaps can persist in the system.

    If you assert that helping your kids raise their test scores wasn’t important, what was important? Be specific, particularly with how you view and utilize content standards.

    I’d like to know what “learning to learn” means to you. I know what it means to me. I teach it within the context of teaching toward content and skill standards, toward facts and skills and helping students reason to understanding. What are your kids learning?

    And I’m not really following you on the last paragraph, as it invokes a couple offtopic strawmen. That there isn’t always a “right” answer doesn’t invalidate standardized tests. That standardized tests be utilized to collect unbiased data with large sample sizes does not imply that students are “learning” through them. The learning happens before and after the standardized tests.

  10. john thompson

    Since this seems to evolving into a conversation I’ll try to respond without overplaying the experience card.

    In this case, intuition based on years of experience versus primitive standardized test data, its not even close. Teaching for understanding IS rocket science. Today’s tests are closer to those of the Wright brothers era. I’d suggest you reread your first paragraph, ask yourself what it is based on, and then ask whether you want to retract it.

    You teach math to younger students don’t you? I have no doubt that today’s tests have more benefits for your field than most others. NAEP results offer confirmation to that common sense appraisal. Even so, I’d be curious how you see things after a few more years of depending so much on data.

    Your career has been during a time when NCLB has concentrated everyones’ minds on deficits. That is still strange to me. Robert Balanz’s recent study of middle schools in Philly says that we should focus more on strengths. Especially with poor teenagers, they need broader engaging challenges. They need successes. This fits my belief that the opportunity costs of focusing on gaps and opposed to building on talents has been damaging.

    To Bill Russell, the father of modern defense in b-ball “learning how to learn” means watching for “triangles.” At times players must focus narrowly and then they must focus peripherally. There are things you can or can’t do while focusing narrowly or widely or switching in between. So he teachs player to “think like a basketball player.”

    I teach high school kids to “think like historians,” like social scientists, like lawyers or like whatever is required by the Standards. I also must teach them to “think like high school students.” They must unlearn the bad habits of middle school of focusing on nothing but the narrow standardized test. (which is doubly sad because middle school should teach the opposite.)

    Data is more than test numbers. A joke embedded in the lesson which requires understanding to produce a belly laugh produces data that can’t be faked. Data happens when I ask about a fourth amendment case that the Supreme Court will address and within minutes everyone is out of their seats and anumatedly discussing and writing on the board the narratives of their 4th amendment cases or their dad’s or their siblings. (or when I explain bypass procedures required by parental consent laws and a student says nobody told her that when she got pregnant by her father raping her and the famale students shifted seats close to her and they basically had their own discussion. I didn’t try to hear their words but I sure learned from their body language.) It would be hard to find an AP class where their purely academic study of the law produces the quality of our scholarly but reality-based analysis. (And you would not believe the personal case studies I hear.)

    I inform my lesson plans and then change them daily based on the NPR reports that evening and in the morning. But I also have repeat benchmarks. When seniors really understand Broad Construction and federalism, those are classes where we can really go for it. When sophomores can link up the social science of colonialism with the social science that informed Brown, then we can take off. The year we elect a black president, I’m supposed to stick with the scope and sequence? For history classes, not ignoring currculum alignment mandates would be educational malpractice.

    Some classes and some years are better than others. So why should I not concnetrate on being honest with myself. Every class has its own personality. Why worry too much about benchmarks?

    Besides, when benchmark testing shows that students are lagging on some topics, the pressure today is to work harder on it. My preferred approach is the opposite. Slow down, don’t over-react, go around the problem , and loop back to it later. Maybe you can’t do that in your field. But I can do it in mine.

    Finally, I listen to students returning from college, the army, jail, or whatever. fundamentally, I earn my paycheck by watching closely, listening, responding, and learning how to read and work with people.

  11. Chris Smyr

    You’ve asked me to retract half-a-dozen paragraphs on this blog, please get over yourself. There’s nothing to retract. You believe your intuition is better than data, so it is no wonder you are so opposed to an influx of accountability in schools.

    In your eyes, besides standardized test data, what else is a “primitive” form of evidence?

    Does focusing on strengths invalidate the use of standardized tests, especially when these tests can help determine what exactly a student’s strengths are in terms of understanding and applying content knowledge?

    Data is certainly more than test numbers, but it doesn’t include “intuition”, especially when your experiences cannot be adequately compared to other teachers’ successes. Using informal assessments to judge learning can be useful, but not if it is the only tool in your toolbox, as informal assessments can lead to biased judgment.

    You already include content knowledge into your teaching along with teaching thinking skills. Standardized test scores might certainly help inform you of additional student successes, which is why I just don’t understand the staunch opposition to them and you telling students that basically these tests don’t matter.

    You instead want to focus on if students have “learned to learn”, which can be a daunting goal to assess uniformly and without equivocating over the definition of such, and that gives little ability to compare results over time and especially between other teachers who have different intuition than you do.

Leave a Reply

Your email address will not be published. Required fields are marked *