Good Reading

While you’re waiting for the election results to come in check out this new paper by Julie Kowal and Emily Hassel about cross-sector lessons on teacher evaluations (pdf). Smart stuff.

Related: Recent takes on teacher evaluation and principal evaluation.

26 Replies to “Good Reading”

  1. So, it will be up and out for the top principals. Its interesting that you use a military metaphor because it betrays the “reform” mindset. Unless you draft principals into the army, your plan would quickly result in an move of top principals out of urban districts into the suburbs and magnet schools where it would be easier to meet performance goals.

    Here’s the fatal flaw in your logic. If only principals did not have to play politics, they could raise test scores. So you fire principals now based on tests scores, believing that politics will disappear. You don’t understand that the imposition of arbitrary high-stakes data-driven acountablity just makes the politics worse. It always has and it always will. T

    hink about it, the ability to enforce discipline and attendance policies are the #1 factor in raising test scores. Show me a real world counter example. That ability, as in other areas of life, is political. As that ability goes to political winners, political losers get dumped with a worse problem.

    The same occurs with teachers. You still seem to believe that accountability can DRIVE reform rather than just be a factor. If that were the case, maybe you’d have some real world examples in the military or some other institution.

    Come on, we’re human beings. Accountability is one part of life. If your vision of human nature would J.P. Morgan or Trotsky or advocates of indutrial policy, or some monopolist or some drug czar or some other top down true believer would have shown lasting results.

  2. That no where in your imagination can you picture a leader who would want a work in a high poverty school or a challenging environment says a lot about you.

  3. John:

    “your plan would quickly result in an move of top principals out of urban districts into the suburbs and magnet schools where it would be easier to meet performance goals.”

    Why are they top principals if they’re worried that accountability will depose them? Surely you can think of a mode of accountability that *doesn’t* find all urban principals lacking?

    “Here’s the fatal flaw in your logic.”

    Hah! John is correcting someone on logic!

    “You don’t understand that the imposition of arbitrary high-stakes data-driven accountability just makes the politics worse.”

    Which is why it shouldn’t be arbitrary (which is the status quo) but purposeful and precise. Why would such accountability make the politics worse? And if schools subsequently improve, does it matter whether there is squabbling along the way?

  4. Yet we lack tests with the psychometric properties and statistical models that underpin growth models to be precise in identifying growth and effective leadership. No one has even developed a model of all the factors that influence a school leader’s impact on achievement, much less created a statistical model that would capture all of those factors.

    For example, the district context within which a principal works influences her/his ability to influence achievement. So, too, does the local supply and demand for qualified and effective teachers. Not only don;t we have any statistical models to control for these factors, we don’t even have ways to collect such information in a way that provides data that would allow us to make valid and reliable inferences.

  5. Billy_Bob:

    The same argument can be raised on the grounds of teacher evaluations– there’s not a model identifying all of the causal relationships between teacher actions and their impact on student outcomes — but it’s made irrelevant by the fact that we can measure student outcomes directly. In other words, we can’t tell for certain what actions Teacher A undertook during the year that caused her students to excel greatly in comparison to other teachers with comparable students, but we still know that Teacher A’s students learned more in the course of a year than other comparable students, and we can thus use this data for evaluations.

    There are similarly also obstacles hindering principals from doing an effective job, and while you have a point that these obstacles are harder to normalize for with principals than they are with teachers, it shouldn’t be considered impossible. If principal evaluations evolve along with teacher evaluations, one can envision a system where principals are evaluated based on their teacher’s successes, each normalized to their respective teacher’s past effectiveness. Has the principal helped his teachers improve over time? If a principal has a lukewarm applicant pool to select from, does he support these teachers to help them flourish? A simpler longitudinal approach to the data could also be an option, where principals are evaluated based on their respective schools’ achievement over time. Issues of supply could be controlled for if it is assumed that the effective applicant pool for a given school has remained relatively static. Has the principal improved the achievement at his school over time? Or are scores continuing to flounder, even though the school down the street is creating gains? Etc..

    Accountability will never be an exact science, just as it’s not in any other workplace. Despite that, there are still functional ways to actually try and incorporate instances of it.

  6. And while we let these models evolve, how many people will lose their job because the models are imprecise? I’m all for exploring these models and using them in formative ways and as one piece of multiple pieces of evidence, but people should never, ever lose their job based solely on s statistical model that is fraught with subjective decisions and error.

    Why can’t policy makers slow down and do things systematically before punishing educators? Their failure to do so makes me think they have ulterior motives other than improving the system. Of course, they all cry that “we can;t wait–we have to save the children NOW!” Yet, by rushing complex policies into the implementation phase in a widespread fashion, the unintended consequences that come with such policies are just as likely to make the situation worse as it is to make it better.

    Again, I implore any one supporting reform efforts to read the history and evaluations of the implementation of reform efforts.

    Ironically, those trying to improve the education system seem reticent to actually educate themselves on the issue before acting.

  7. “I’m all for exploring these models and using them in formative ways and as one piece of multiple pieces of evidence, but people should never, ever lose their job based solely on s statistical model that is fraught with subjective decisions and error.”

    Sounds like you and Chris are in agreement. Glad to hear we’ve resolved this debate.

    “Yet, by rushing complex policies into the implementation phase in a widespread fashion, the unintended consequences that come with such policies are just as likely to make the situation worse as it is to make it better.”

    Please give us an example where a serious person proposed instantly adopting a statistical model as 100% of a teachers evaluation?

    Also I understand that you’re replying to the comments on this thread, but did you actually read the report? I bet you’d like their proposals

  8. I’ll add a little more to Dick’s comments.

    “And while we let these models evolve, how many people will lose their job because the models are imprecise?”

    The precision argument is poor considering that the confidence intervals (for VA for instance) can already be brought to low enough levels to distinguish between great and poor educators. We could likely see the same usage for principals.

    “Why can’t policy makers slow down and do things systematically before punishing educators?”

    Because I imagine that, were it up to you, you’d ask for 100% precision before any such accountability measures were put in place, essentially “dooming its chances of failure” before even being implemented. There doesn’t need to be 100% precision.

    “Again, I implore any one supporting reform efforts to read the history and evaluations of the implementation of reform efforts.”

    You’ve cited vague historical accounts before. Care to offer some specific examples to help bolster your argument here?

  9. “The precision argument is poor considering that the confidence intervals (for VA for instance) can already be brought to low enough levels to distinguish between great and poor educators. We could likely see the same usage for principals.”

    I don’t know where you get this, but is is clearly incorrect. Read the NECS report on this or read school finance101.

    As for principals–you didn’t understand my post (gee, why am i not surprised). So, we are going to hold principals accountable for student achievement when principals do not control their work environment? Sounds like a recipe for disaster.

    No–I would not ask for 100% accuracy. I would require far, far more than existing VAM models can deliver. I would pilot it in a few places using a variety of models and not fire people based on flawed info, but would use it in a formative fashion.

    Read Tyack & Cuban, “Tinkering Towards Utopia
    for a start. I would read all the Tyack and Cuban pieces as well as Joel Spring and Diane Ravitch. Cremin has some great pieces as well.

    One issue they cover is the complete and utter failure when business people get involved in ed reform. And they are making the same mistakes as made before–because they don’t understand history and don;t understand how teachers think and work.

  10. “I don’t know where you get this, but is is clearly incorrect. ”

    Most of what I’ve read give examples of models that have 30% intervals of error. Explain why that is too imprecise to resolve the differences between educators in the top 20%, let’s say, and the bottom 20%.

    “As for principals–you didn’t understand my post (gee, why am i not surprised). So, we are going to hold principals accountable for student achievement when principals do not control their work environment? Sounds like a recipe for disaster.”

    I understood it well enough, but there’s not much useful that can be teased out. What exactly do principals not control that can’t also be normalized out with a clever model of evaluation? I gave a couple examples for how this could happen.

    “No–I would not ask for 100% accuracy.”

    Good.

    “I would require far, far more than existing VAM models can deliver. I would pilot it in a few places using a variety of models and not fire people based on flawed info, but would use it in a formative fashion.”

    *sigh*

    1) What does “far, far more” look like? Be specific.

    2) You are begging the question of why we should assume current models function on “flawed info”. WHY is it flawed? Is it because it’s not precise enough?

    3) Considering that’s presumably what anyone wants to do right now — “pilot it in a few places using a variety of models [… and] use it in a formative fashion” — I get the feeling you come here just to disagree with whatever Andy posts.

    “Read Tyack & Cuban, “Tinkering Towards Utopia
    for a start. I would read all the Tyack and Cuban pieces as well as Joel Spring and Diane Ravitch. Cremin has some great pieces as well.”

    Do you not understand what “specific” means? I’m not asking you for books to read, so someday I can try to tease out the points that I think *somehow* bolster your argument. Give the actual historical examples that bolster your argument.

    “One issue they cover is the complete and utter failure when business people get involved in ed reform. And they are making the same mistakes as made before–because they don’t understand history and don;t understand how teachers think and work.”

    This is not evidence suggesting that increased accountability within principal evaluations is a bad idea. If anything, that you’re implying that such reform is a bad idea because business people support it only suggests you’re not going to win this debate, either.

  11. I have no doubt that soon the general public will see the truth of the present “reform,” if they don’t already, as descibed by Larry Cuban and other experts.

    We CAN and SHOULD have real reform for urban schools but that reform must begin by acknowledging the truth: Education is a complex process requiring cooperation of students, parents, teachers and communities. There are no simplistic solutions. Once we accept this obvious fact, we’ll begin to see real progress.

    Also, Cuban reminds us that most Americans get a good education from their public schools. If you doubt what he says, think of your own children and those of your neighbors, friends and relatives. If you are still very young (e.g. Chris) think of yourself.

  12. This is pathetic, Billy.

    “It is also a myth that all U.S. schools are broken. Not only I but many others have pointed out that the U.S. has a three-tiered system of schooling where the top two tiers have mostly “successful” schools by current standards. The bottom tier contains failing urban schools. Thus, all U.S. schools are not failures by any standard.”

    Not all schools are failures, just the schools that are failing. LOL

    “Pay-4-performance schemes and the use of test scores to evaluate teachers also have been critiqued severely by experts who are far more familiar with the innards of these “reforms” than the champions who promote they [sic] ceaselessly.”

    Lots of stuff has been critiqued here, too. That doesn’t mean they are particularly good critiques. Ravitch’s twitter, for example, is a distillation of all that is horribly wrong with this comment board.

    “And “no excuses?” The departure of Chancellor Michelle Rhee from Washington, D.C., may signal the beginning of the end of the simple-minded view that teachers and principals, by themselves, can erase the ill-effects of poverty, inattention to children at home, and neighborhood pathologies by setting high expectations and academic standards.”

    Yes, because if we have kids in poverty, then it’s pointless to hold high expectations and academic standards. But of course, this is not a defeatist argument to maintain the status quo of failing schools, oh no!

  13. Linda:

    “I have no doubt that soon the general public will see the truth of the present “reform,””

    Does that truth involve reformers lining their pockets at the expense of children?

    “We CAN and SHOULD have real reform for urban schools but that reform must begin by acknowledging the truth: Education is a complex process requiring cooperation of students, parents, teachers and communities. There are no simplistic solutions. Once we accept this obvious fact, we’ll begin to see real progress.”

    I love how this is supposedly an argument against having higher accountability in schools.

    “Also, Cuban reminds us that most Americans get a good education from their public schools. If you doubt what he says, think of your own children and those of your neighbors, friends and relatives. ”

    Linda, if someone claims that “most Americans get a good education”, the first response you should have is NOT to “ask your neighbors, friends and relatives” if it’s true. You’re assuming here that the sample of your closest friends is representative of all Americans, and I can guarantee you that’s not the case.

    The achievement gap is a problem that exists throughout our education system. That it doesn’t directly affect you or your neighbors seems to make you think that it’s not important enough to consider noting.

  14. The achievement gap most assuredly exists and we must do everything possible to close it. I spent 42 years trying and that’s why I understand how complex it is.

    For many years teachers have been trying to close this gap by (first of all) being in the classroom but also by fighting for resources that poor children lack. (Did you know that the average teacher spends more money on her classroom than the federal government?) However, there are powerful forces in the United States fighting to keep the status quo, which I define as keeping resources low for urban children and (worse) siphoning off these scant funds for private gain. Fortunately most citizens do understand that teachers need help in educating all children and will support them in trying to destroy the status quo that keeps so many children down.

    None of what I have just written negates the fact that about two-thirds of our students get a good education in our public schools. Our problem is how do we make sure that ALL students have access to this. Please join parents and teachers in their efforts to improve education for disadvantaged children.

  15. If anyone is confused about the agenda of the present “reform” movement, just look at Washington, DC, where Michelle Rhee gave everyone who cares to look a good insight into the true intent of her type of “reform.”

    She acknowledged the importance of the quality of the classroom teacher, a fact that everyone seems to agree with. Backed by philanthropists and the mayor, and with the additional advantage of the recession, she had the golden opportunity to comb the country for some of the most experienced and successful teachers. She could have hired people who would have stayed for years in DC, bringing thousands of children high quality instruction. Instead she hired mostly young people right out of college. Not only that, but many of these new recruits had no intention of staying more than a couple of years. Now why did she do that?

    The answer might be related to the fact that these teachers came from agencies with ties to Ms. Rhee and her friends, relatives and associates. These agencies were paid thousands of dollars for each new teacher.

    To me, that pretty much sums up “reform” at the present time. I’m proud to say that I was not fooled, nor were the citizens of DC. Soon the rest of the country will know.

  16. Linda:

    “However, there are powerful forces in the United States fighting to keep the status quo, which I define as keeping resources low for urban children and (worse) siphoning off these scant funds for private gain. ”

    Be specific: what are these ominous forces?

    “None of what I have just written negates the fact that about two-thirds of our students get a good education in our public schools.”

    1) Just because they are white doesn’t mean they are automatically receiving a good education. Having your friends confirm your thoughts doesn’t make it so, either.

    2) How does this have anything to do with principal evaluations?

    “Our problem is how do we make sure that ALL students have access to this.”

    One such way that we’re discussing in this thread is expanding the accountability in schools to better encompass principal evaluations. You are welcome to join in the discussion.

  17. that’s a pretty bold claim Linda. Do you have any evidence that the number of TFA or TNTP recruits went up under Rhee’s tenure? I’m not denying that it may have but i’d like to see you support your assertions with facts.

  18. Linda:

    “she had the golden opportunity to comb the country for some of the most experienced and successful teachers.”

    1) How should she have gone about combing?

    2) How should she have enticed teachers to move to DC after she combed?

    3) What do you think the communities at the schools she took these teachers from would have said about such?

    4) Why do you suppose teachers in distant reaches of the nation would automatically be a good fit in DCPS?

    “Instead she hired mostly young people right out of college. Not only that, but many of these new recruits had no intention of staying more than a couple of years. Now why did she do that?”

    1) You’re making assumptions about the motives of young teachers that you can’t support. Stop it or piss off.

    2) Considering your above strategy of “combing” has several major flaws in it, hiring young talent might be a more realistic fix.

    3) There are several lines of evidence that suggest young talent taken from good prep programs are just as good as other teachers, likely making Rhee’s decision a good choice given her options.

    4) I’ve not even mentioned the simple likely fact that there weren’t many excellent veteran teachers just waiting to get picked up.

    “The answer might be related to the fact that these teachers came from agencies with ties to Ms. Rhee and her friends, relatives and associates. These agencies were paid thousands of dollars for each new teacher.”

    You are disgusting:

    1) Give the evidence that suggests Rhee hired only from agencies she is affiliated with.

    2) Give the rationale for why Rhee would want to only hire teachers from groups she is affiliated with (what specifically does she have to gain?).

    3) Give the counterarguments as to why Rhee shouldn’t have hired new teachers.

    4) Give the evidence that suggests Rhee is generally not interested in helping better education but rather in helping herself (you continue to imply this but NEVER can justify it).

    5) There’s no grounds for assertions that Rhee manufactured a budget crisis to lay off other teachers, nor is there evidence that she let go older teachers just so she could hire newbies, so be sure your answers do not include this.

    (https://www.eduwonk.com/2010/10/please-pay-no-attention-to-the-evidence.html )

  19. I give up. And everyone else should too. Chris will NEVER admit that anyone else has anything useful to say. He is a pathetic, basement-dwelling dweeb who uses this blog to feel as if he is important. I refuse to waste any more of my time trying to argue with someone who simply refuses to listen. Its pointless and we all have better things to do with our time.

    Chris will think he are weenies and he is the king for us not engaging in argument, but he already thinks that any way. Chris may think he is helping, but just makes people see all reformers as complete idiots who refuse to listen. It is people like Chris who will bring down their effort.

    So, Linda and edconsumer–just give it up. Dont engage. Let Chris find other things to do with his time while in hos parents’ basement.

  20. Billy Bob:

    Far be it from me to impose on your rantings, but:

    1) You have never finished a debate that you’ve engaged on this blog. Ever! And it’s not because I was mean or nasty to you. Look above: the last relevant comment you posted in this thread asked me specific questions which I answered, and it brought up vague counterarguments which I explicitly responded to, questioning their validity and meaning. Surprise, surprise– you gave no response.

    2) You then posted a link to another blog post that failed to respond to my questions to you, nor was it relevant to this discussion on principal evaluations, nor was it even thought-provoking on its own merits. Surprise, surprise– my postulated law of Eduwonk debates holds again. (https://www.eduwonk.com/2010/08/kipp-and-catholic-schools.html#comment-208856 )

    3) Linda butts in with a well-timed infantile remark, accusing Rhee and reformers as scurvy edu-pirates, motivated by greed, intent on pillaging schools for their own gain. Will she buck the trend and support her outrageous claims? Until she does, “hide yo’ kids, hide yo’ wife,” just in case Andy is one of them.

  21. “You are disgusting.”

    This is the kind of “teacher” the poor kids get. I rest my case.

  22. Chris is undoubtedly a real jag, but you hardly ever address the points he lays out. When you do, he responds promptly with generally decent (if sarcastic) rebutting.

  23. I’m a jag for being sarcastic about terrible and slimy arguments? Sarcasm and mockery are the only things keeping the rebutting fun around here.

Leave a Reply

Your email address will not be published.