Pass/Fail

Today’s NYT has a look at the new teacher evaluation system in D.C. Three things jump out.

Landscape. The headline is “Pass or Be Fired.”  But, while that is the choice in some places – and teacher are understandably resentful –  is that really the choice in D.C.?  The evaluation system is part of a revised compensation that offers teachers the chance to earn substantially more for performance, challenging assignments, and so forth.  So the overall deal is more – demand more and pay more – than just pass or lose your job.

Context. This passage is interesting.  ““I’m 25 years in the system, and before, I always got outstanding ratings,” she said. “How can you go overnight from outstanding to minimally effective?”” Good question!  Perhaps this is where you key readers in that everyone was rated outstanding before yet somehow the kids were not learning very much?  It’s not a theoretical question, there is data on multiple school systems and this is why as part of the 2009 Recovery Act the feds started collecting data on evaluation outcomes.

Process. In education we always fight about process…Here’s the kicker:

“Some teachers feel the master educators are out to get them.” That is a common perception, said Mark Simon, an education analyst for the Economic Policy Institute, which receives teachers’ union financing. Ms. Rhee developed the system, he noted, during tough contract negotiations and did not consult with the teachers’ union in its design. “That was a missed opportunity,” Mr. Simon said, “and it’s created a lot of resentment.”

Well, the union did ratify the contract that included all this –including decoupling evaluation from the contract. They did so overwhelmingly it should be noted.  Might also be good to note that Simon was a teachers union leader, leader as in president, of the union right next door to D.C. in Montgomery County.  There is some sibling rivalry here…especially around the issue of “collaboration.” But all that not withstanding, at what point should we just start looking at the evaluation system on its merits (certain aspects of which are debatable it should be noted)?

19 Replies to “Pass/Fail”

  1. As the article points out, teachers in low income schools teaching low ability, low incentive kids are always going to be reamed more. The teacher in a high ability school doesn’t have to worry about kids eating or texting in a classroom–she calls the parents, the kid gets grounded if it happens too often. The teacher in the low ability school who tries to focus on teaching and doesn’t notice every single discipline problem gets reamed.

    The Times had another story a while back on a math teacher who had to completely redo his lesson on the fly because none of his kids knew a basic skill needed for that lesson. He got dinged for that.

    This is always the case, by the way. Of all the people I graduated with from ed school, the few who have already given up teaching 3 years in are the ones who got handed difficult classes in Title I schools–teaching algebra, teaching reading labs, and so on. And they all lost job after job for classroom management. They would have been completely fine with your average classroom of suburban kids–indeed, the classmates who are in those schools are doing just as well, with no better skills, and making more money.

    IMPACT and other assessment tools actually penalize teachers who want to improve kids. But of course, in a few years, it will show this and people like the Wonk here will have moved on to something else.

    Note: I am not arguing in favor OF anything. I am merely pointing out that evaluating teachers with low incentive, low ability kids can’t be done fairly with the same rubric used to teach high ability kids.

  2. Cal: I agree with your post (above). In particular, I liked this section: “Of all the people I graduated with from ed school, the few who have already given up teaching 3 years in are the ones who got handed difficult classes in Title I schools–teaching algebra, teaching reading labs, and so on. And they all lost job after job for classroom management. They would have been completely fine with your average classroom of suburban kids–indeed, the classmates who are in those schools are doing just as well, with no better skills, and making more money.”

    When I worked as a teacher (before changing to a career as an attorney), I found it was MUCH easier to teach the highly motivated (often high-income) students than to teach the low-income students, who often came to school unprepared to learn. I eventually quit the profession after being placed with an extremely difficult group of students in a low-income middle school and being harshly (and unfairly) evaluated by the principal as I struggled to teach in this challenging situation.

    Holding teachers of low-income students from broken families to the same standard we hold teachers of highly motivated students from supportive families is outright inappropriate. The result will likely be teachers, like me, who simply leave the system.

  3. Because I moved around the country a lot during the early years of my career, I had the opportunity to teach in very different kinds of schools. In the private school that catered to affluent children, the test scores were often above the 90th percentile for nearly every child. Even in the classrooms that had subs the whole year, due to a teacher on sick leave, the children achieved at a very high level. In the “inner-city” of Cleveland, most students scored below (yes, below) the tenth percentile and in Iowa, where I taught in an “average” school, the scores hovered around the 50th percentile.

    We’ve known for over 50 years that standardized tests, when administered correctly, correlate closely with the socioeconomic status of the child. We can continue to ignore this, but it won’t help the children. To improve education, we must ask this question: What does the advantaged child have that helps him to achieve? How can we provide some of those services and experiences to all children?

  4. “We’ve known for over 50 years that standardized tests, when administered correctly, correlate closely with the socioeconomic status of the child.”

    True enough. But we also know from mounds of research (since the days of Bill Sanders at UT) that the actual growth or improvement of students is not well correlated with socioeconomic status. In short, some schools and some teachers “get more” learning growth out of their students REGARDLESS of student background.

    No one is arguing that the baseline test scores of rich kids is what all teachers should be judged on in evaluation. Of course student background matters there. What folks are saying is that the quality of teaching varies and that it – quality teaching – has an impact on the improvement kids make irregardless of their background. The degree to which educators can induce improvements ought to be taken into account in a well designed evaluation system.

    Now, the devil is in the details in terms of how many years, observations, etc are needed to derive a reliable growth measurement of a teacher’s quality. I’ll leave it to the stats guys to figure that out. But don’t tell me it can’t actually be done. Human resource organizations in near ever profession have some degree of uncertainty in their measures of employee evaluation. What makes public education employees so immune from some error in the way evaluations are carried out? I guarantee you value-added growth models A) would never have been pushed if unions hadn’t whined about giving autonomy to principals to make evaluation decisions thereby creating a demand for something less subjective and B) are far more objective than the review Joe Blow gets when he walks into his manager’s office at the end of the year to talk about his performance.

  5. Come on, you know IMPACT isn’t fair for high poverty schools. And DC is a high poverty district. Obviously, its about firing people. Obviously its about getting rid of Baby Boomers and veteran teachers. Its about destroying the “status quo.” Veteran teachers are the status quo, and its a fig leaf to get rid of us so they can start with a clean slate and socialize newbies into their system.

    You know that. True believers in the politics of destruction don’t hide their light under a bushel. After all, if they can drive veteran teachers out of the district, its easier for them.

  6. daprofessor:

    Of course, some teachers and schools are better than others. That’s just common sense.

    During my many years as a teacher, it never even occurred to me that I wasn’t being evaluated on the basis of student achievement. The competent principals came into my room frequently, observed student work and had access to student files. In my state a principal had full access to files and could look at standardized test scores or anything else. It was his or her job to know whether or not the students in my class were learning.

    Some teachers liked to brag about the progress of their students. I’m embarrassed to say that I was one of these people. I’d send papers to the office, ask principals and other teachers to hear my students read and ask the superintendent how my kids compared to those at the affluent school across town. He always said my students did just as well and I believed him.

    I never heard that teachers didn’t want to be “accountable” until the “reform” movement. What does this mean? My guess is that teachers do not want to be evaluated on one test that the students take. Would you? Let me explain why, based on my own experience:

    When I was a fifth grade teacher I had a boy in my class (“Jose”) who had been in the United States for a little over a year. Jose had never been to school in Mexico and could not read or write in Spanish. At the end of fourth grade Jose had scored in the seventh percentile on the standardized test.

    The instructional aide, the reading teacher and I got to work on Jose. We tutored him each day so by the end of the year he could read and write at a third grade level. He was thrilled and so were his parents. In May of that year I was required to include Jose in the standardized testing because he had been in the country so many months (can’t remember exact number). Because the test was geared to the end of fifth grade, there was almost nothing on the test for Jose. He scored at the fifth percentile. Was I a “bad” teacher for Jose? I didn’t think so and neither did he, his parents or the school principal. I had brought him to literacy. He could now speak, read and write in English but, no, he could not do it at the same level as native English speakers. Does that surprise anyone? (By the way, Jose came to visit it me years later. He was in college by then and thanked me for teaching him to read. He said I was his favorite teacher.)

    As other posters have mentioned, the likely purpose of the present “accountability” system is to get rid of the higher paid veteran teachers. Most people are sharp enough to know that you can’t evaluate a teacher with a whole class test or five short visits to the classroom. However, it is NOT difficult to do an evaluation that is thorough and fair. In order to do so, the evaluator would have to:

    be aware of EACH child’s level of achievement in the fall;

    be aware of extenuating circumstances (child was absent half the year because he went to Mexico when his grandmother died);

    be aware of the progress of the students throughout the year by examining their work as well as benchmark tests;

    observe the teacher frequently;

    be familiar with the level of satisfaction in regard to students and parents.

    administer a post-test to EACH child at the end of the year. A group test would suffice for children who are at or near grade level, but children who are significantly below or above grade level would need to be tested individually.

    Because many districts will now be tempted to get rid of Old Mrs. Brown to make room for Young Mr. Smith, I suggest that teachers protect themselves by doing the following:

    Give a careful pretest to each child in September. Include a sample of the child’s writing (composition). Ask another teacher, the principal or a parent to be a witness. If allowed, record the child’s oral responses and reading.

    Keep careful records throughout the year. This would include benchmark testing and samples of student work. Keep the principal informed at all times and ask him or her to respond to the work in writing. Again, have witnesses.

    Ask the parents to evaluate the work of his child.

    Give a post-test at the end of the year. Make certain it is well proctored. Better yet, ask another teacher or the principal to administer the test.

    Make certain parents are aware of the progress of their children. They make excellent witnesses.

    If you are suddenly deemed “ineffective” after 25 years of “excellent” you now have a lot of evidence to support your excellence as a teacher. Good luck!

    (If you are at the beginning of your career, try to get a job in an affluent district until all this nonsense passes. If you are unable to, consider subbing in such a district until there is an opening. If a teacher quits abruptly, districts often hire someone who can take over immediately and with whom they are familiar.)

  7. From a principal in DCPS:
    He said he is quitting a system that evaluates teachers but doesn’t support their growth,

    Kerlina signed on just as Rhee was rolling out the IMPACT evaluation system, …
    It was a major change.Kerlina said he was surprised when he heard it would not be tried on a pilot basis, which was standard practice in Montgomery. He said he came to believe that the initiative offered virtually no provisions to help teachers improve.

    “The reform, in my opinion, is getting rid of people,” he said.

    http://www.washingtonpost.com/local/education/young-dc-principal-quits-and-tells-why/2011/06/19/AGfcP6kH_print.html

    Response from the school system:
    “I guess that we all know everything ain’t for everybody,” Chanchellor Kaya Henderson said.

  8. I should have pointed out that the students of Montgomery County do a much better job than the studnets of DCPS on tests so beloved by the data-queen.

  9. “The reform, in my opinion, is getting rid of people.”

    Well, yes. And it should. Anyone who has gone through DCPS as a student (as I did) knows full well that there are MANY teachers who need to be fired — and should have been fired years ago. That’s the uncomfortable truth that underlies everything. Because of a systematic lack of accountability, some awful teachers probably could make it 25 years with “outstanding” ratings even though in reality they are “minimally effective” or worse.

    IMPACT is not perfect — there is no perfect system of evaluation and never will be — but at least it sends the message that someone is going to hold teachers accountable.

  10. Principal Kerlina was not referring to those who need to be let go but rather to those who could use help and assistance but won’t get it, so time and money will be spent on hoping the next person hired doesn’t need assistance.

  11. to borrow a quote from caddyshack (the judge says to ty webb: “if you don’t keep score, how do you measure yourself against other golfers? ty (chevy chase): “by height”

    if we aren’t going to evaluate teachers based on their students progress academically, what ought we to evaluate them on… height?

    seriously marlowe…

    what should we evaluate teachers on when it comes to granting promotions, boosting salaries?

    effort? years of experience? masters degrees? what? and after you answer that, tell us how your “rubric” is in the best interest of children and the public.

  12. daprofessor,
    The test results show that teachers in MD don’t need their evaluation tied to test scores.
    What should they be evaulated by, you ask?
    Whatever the 24 districts in Maryland are already using.
    Obviously, except to you, that has been working.

  13. Working? So there are no differences in teacher effectiveness across the teacher labor force in Maryland? There is nothing to measure in terms of some teachers being high performers (able to get more learning gains than others) and others being low performers? Great, now you’ve solved the 64,000 dollar theorem in human resources. You’ve actually found an organization where all labor inputs perform at the same precise exact level of productivity. Marx would be impressed. So of course no need to differentiate since all teachers are the same in terms of productivity. Widgets I tell you! Not professionals. Bring back the Progressive era 1930s baby!

  14. Thank you daprofessor.
    I’m glad your reading class at Fillmore Bush High School helped you.
    In a half a day at that!

    The only high performer teacher in Maryland’s history was Michelle Rhee.
    She took her second grade students from the 13th percentile on the CTBS to 90% of them (63 in number) scoring in the 90th percentile as third graders.

  15. K-12 teaching is the most self-selective of all the professions. About 50% of all teachers quit during the first five years. Most do so of their own volition (It’s an almost impossible job if you’re incompetent or you don’t enjoy it) while others are counseled out or pushed out. Some do not have their contracts renewed. Almost none of these teachers is “fired” because they have contracts. In order to find out who was “dismissed” an investigator or researcher would have to do a lot of digging. For example, the board minutes for my former school district include many “resignations” each year. I know that some of these people were asked to resign or were “nonrenewed” but it doesn’t show up in print because the teachers were allowed to “resign.” If a teacher is “terminated” (very rare) it’s usually because a crime or serious infringement (job abandonment) has occurred. I believe this is the source of the myth that a teacher can’t be fired.

    So for those of you who want “accountability” in teaching, you have it to a much greater extent than in any other profession because teachers can also be dismissed for “moral turpitude.” What we need is a method for attracting and retaining qualified individuals to K-12 teaching. Retaining teachers, especially in urban schools, has been a huge problem for many years. Even during this recession, places like DC have a very difficult time hanging on to teachers, principals and superintendents.

    We are having the present discussion because of the effects of the Great Recession. Districts are desperate to unload expensive ” Mrs. Brown” so they can hire two “Mr. Joneses.” Our legislative and executive bodies are hugely affected by corporate interests but the courts are still relatively indepent. I believe the “Mrs. Browns” of the country will find vindication in our courts as well as the court of public opinion.

  16. Linda:

    What data are you looking at when you make the claim that:

    …”teaching is the most self-selective of all the professions. About 50% of all teachers quit during the first five years. Most do so of their own volition…”

    The only research I know on this subject is a recent paper by West and Chingos (2010) that looks at high-quality micro data from Florida tracking teachers who leave teaching versus those who stay. They find (the money quote from the article) that:

    “A clear majority of the state’s most effective teachers leave their initial schools only four years into their careers, and these same teachers are no less likely than the least effective to leave the state’s public schools system altogether.”

    Full cite:
    “Teacher Effectiveness, Mobility, and Attrition in Florida,” with Martin R. West, in Matthew G. Springer, ed., Performance Incentives: Their Growing Impact on American K–12 Education, Brookings Institution Press, 2009, pp. 251–271.

  17. I see this statistic mentioned frequently. Just google it and you’ll see many references to it.

    Thanks for mentioning the Florida study. That’s my impression also.

  18. Again quoting “Teacher Effectiveness, Mobility, an Attrition in Florida”…

    “The value of the present analysis and findings hinges on the assumption that the test instruments used are, on average, a reasonable
    measure of students’ overall academic development.”

    “While the description of career paths by effectiveness tercile is straightforward, the estimation of a credible value-added measure is more challenging. There is little consensus among researchers who study teacher quality as to the preferred empirical specification of the relationship between test scores and student, teacher, and school characteristics.”

    “The model, which is similar to that employed by Kane et al. (2006) in their study of New York City teachers, is: delta A(it) = beta X(it) + gamma C(it) + phi S(it) + delta W(it) + pi(it) + epsilon(it) , (1) where delta A(it) is the decile-standardized test-score gain of student i in year t; X, C, and S include student-, classroom-, and school-level characteristics; W is a vector of teacher experience dummy variables; pi is a vector of grade-by-year fixed effects, and epsilon is a standard zero-mean error term. We estimate equation (1) separately by subject (reading and math) and test (FCAT and SAT) and average the residuals by teacher and year to construct four value-added measures of teacher effectiveness. The key confounding factor in the estimation of the impact of teachers on student achievement is the non-random matching of students and teachers that occurs both across and within schools.

    And from the paper’s conclusion:
    “When we compare the mobility rates of the most and least effective teachers at specific schools, we find that the least effective are actually somewhat more likely to leave in the first years of their careers. And schools with high-performing students are able to retain a bare majority of their most effective teachers while dismissing all but a handful of their worst performers.”

    Attorney DC:
    I don’t believe any of the variables listed in the value added equation considers the motivations and behaviors of the extremely difficult and challenging middle school class you struggled to teach and then were harshly evaluated on. Maybe the reason so many teachers in the study left within four years is exemplified by all the support you received from your principal with that challenging class. Then again, maybe these invisible student parameters which can make one classroom so different from the one next door or one period so different from the other periods the same teacher teaches can be linked to those unobservable poor instruction practices in a future value added model.

Leave a Reply

Your email address will not be published.