Teach For America And The Problem Of Study Laundering

I mentioned some sloppiness around the the recent report about Teach for America (TFA) that Michael Winerip featured in his column from a few weeks ago in an effort to make the point that the research on TFA is mixed.   Since we seem to be repeating history now seems a good time to revisit that and the larger issues it raises. The report conveniently highlights two problems:  Our field’s pathetic and weaponized approach to research and the problem of “study laundering.”

Pile ‘em up: The two big takeaways of this report from the Great Lakes Center is that retention of TFA teachers is bad and the program’s results are, at best, mixed.  There are substantial problems with both findings.

On the retention issue the researchers seem to be focusing on whether Teach For America Teachers leave their schools after two years, not whether they leave teaching. Unfortunately, this is a common mistake in research on teacher attrition especially when the goal is to illustrate bigger numbers (for instance all the research about how the attrition of new teachers is so far out of line with other fields).

The Great Lakes Center report states that “(M)ore than 50 percent of TFA teachers leave after two years, and more than 80 percent leave after three years…”  The report reaches this figure by consolidating findings from previous studies that in one way or another conflated leaving a school with leaving teaching.

In fact, in a study that delineated the leaving issue more effectively, a 2008 study by Harvard’s Project on the Next Generation of Teachers, found that 61 percent of Teach For America corps members stay in teaching beyond the two-year commitment.  Teach For America surveys its alumni regularly and the most recent survey found that 65 percent of Teacher For America’s 20,000 alumni remain in education, with 32 percent continuing as teachers.    And remember, that’s a survey of alums going back almost two decades now so that one in three figure should be viewed in that context as well as the larger context of TFA’s mission.

On the question of aggregate TFA performance the report also falls short.   There are research methods and they’re not equal in terms of analytic leverage.  All the commentary attempting to present the case of mixed effects for Teach For America teachers succeeds only by piling up all the studies and then saying, huh, two big piles so the studies are mixed.  In fact, if you look at the studies that employ the most rigorous methodology (in other words, apples to apples, enough apples to make a reliable estimate, etc…) it’s pretty unambiguous that, as a group, Teach For America teachers perform as well or better than other teachers, not only emergency certified teachers but traditionally trained ones and veterans.  Considering that on an annual basis Teach For America is now the largest teacher prep program in the country (excluding multi-campus ventures such as the UC system) that overall level of quality is a big deal.

For instance:

A 2004 study from Mathematica Policy Research found Teach For America corps members were as good or better other teachers, including veteran teachers.  This was the only study to earn an A for its methodology in a 2008 Ed Next analysis of research into Teach For America because of its methods.

A 2009 Urban Institute study that found the impact of having a Teach For America teacher was at least twice that of having a teacher with three or more years of experience.

A 2010 study from the University of North Carolina, which concluded that students taught by corps members outperformed their peers in high school science, math, and English. At every grade level and subject studied, Teach For America corps members’ students performed as well as or better than the students of traditionally prepared UNC graduates.  This was a state study to help inform policymaking there.

Other studies here.

This doesn’t mean that TFA teachers are all outstanding.  There is high-variance amongst them, just as there is with other routes into teaching and TFA teachers struggle their first year, just as most teachers do.  But these results do mean that in the aggregate hiring a Teach For America teacher is a pretty safe bet, relative to all the other options on the table.  This is part of a larger body of research on teacher effectiveness that shows that  – outside of emergency credentials with no training at all – routes into teaching matter less than candidates.

And the secondary impact is in no small part fueling today’s reform movement (pdf).  Former corps members are all over the place.

TFA critics continue to cite the David Berliner study on TFA from 2002 as evidence of TFA’s “mixed results.”  Sorry.  Here’s a review of that study by Kosuke Imai (pdf) and here’s a more accessible review by UVA’s Paul Freedman (pdf). As both make clear, the Berliner study wants for rigorous methods: Before you even get to the statistical sleight of hand, which isn’t that complicated to ferret out, the selection problems undermine its methods.  That’s why the 2008 report card gave it a ‘D.’ Punchline:  Not all research is created equal.

Again, to date no study with what would be considered rigorous methods (meaning adequate controls) has shown that Teach for America teachers depress student achievement.   That’s noteworthy but lost in the noise. On some issues (eg charter schools) the research is mixed.  That’s really not the case with Teach For America right now.

Study laundering: I know it’s impolitic to forthrightly point this out, but here’s the deal:  In terms of mainstream media, only Winerip and, of course, Mikey bit on this study despite that it had been shopped around for some time.

That’s in part because the board of the center is made up of people with a track record of trashing Teach For America and NEA affiliates fighting to keep TFA out of various states.  That’s all fine, I’m a big fan of the five freedoms.  But, most reporters would (and did) then take a critical eye to the findings.  Perhaps ask some disinterested researchers to have a quick look at the studies being aggregated?  Yet not here.  Rather: Hook, line, sinker.   If  Winerip and Mikey covered tobacco research, we’d all still be taking cigarette breaks during the workday.

So what happens is that the Great Lakes Center puts out the study, no one serious bites.  But then it ultimately does get picked up, for whatever reason, and – voila! –  it’s clean money! In other words, suddenly it seems more legit because it earns the moniker  ‘as reported in the Washington Post’ or ‘this work was featured in the New York Times.’  This happens will all kinds of studies, pro-and anti-reform, by the way, and it’s a big problem that confuses rather than clarifies things for the casual observer or the policymaker trying to make heads or tails of an issue. In other words, the problem of the easily fooled or the agenda-driven becomes everyone’s problem because it further clouds already complicated issues.

Update: Professor Berliner responds below. He throws up some misdirection (it was peer reviewed!), attacks the reviews, but admits the criticisms have merit, and then unfortunately fails cite any specifics or say which ones.  That’s a problem because the criticisms undermine the premise of the study.   To quote from Paul Freedman’s* analysis (pdf), again the more accessible of the two reviews and only a few pages and worth reading, the three issues are:

· problems of selection and inadequate matching fundamentally undermine the validity of the study;

· the authors overstate the substantive importance of their estimates;

· the statistical approach employed is not well suited to the research question.

Rather, Professor Berliner argues that, ” But the study we did with very careful matching procedures met some of the standards of quality that the profession had for conducting non-causal designs.”  Given the growing body of research about Teach For America that meets more than “some” of the standards, and in fact allows for causal inferences, that statement is an excellent summation of the problem here.

By the way, here’s a bit on Freedman, who doesn’t even have a dog in this fight.

91 Replies to “Teach For America And The Problem Of Study Laundering”

  1. Chris,

    Thank you for your thoughtful reply.

    Your story cannot be misleading, if it is true and was what you experienced.

    You certainly don’t have to speak for TFA, you only have to be true to your narrative. That will be enough in this context.

    I can guarantee you that all the teachers and retired teachers on this comment board of just being truthful to their personal narrative and their experience of public education in this country.

    Any animosity that has bubbled up here stems from the gulf that exists between telling your story and believing that you are needed to tell the “right” story, or TFA’s story, or the reformers story.

    Democracy is the stories of all citizens woven into a coherent public program. That is what has always been at the heart of great public education – and when we stop or won’t tell our stories we lose the public narratives that sustain change and real reform – not the reform pushed by political elites and connected philanthro-capitalists.

  2. John Ciden: “Nevertheless, can someone explain why – even assuming that only 25 percent of former Teach for America teachers remain teaching after 4 or 6 or 8 years – if they are good teachers and would not have been teaching without Teach For America, why is that bad?”

    No, that’s not bad, if that’s what it is. Again, my main issue was how Rotherham was spinning the facts. It was he who seemed to find something wrong with a study finding that the percentage who stay in teaching was low. I can’t imagine why he felt compelled to correct that. Maybe he thought it cast a bad light on TFA?

    Here’s another take: Rotherham cites TFA stats that 32% of all 20,000 alumni are still teaching. Yet the Harvard Study that he cites finds that less than 30% continue teaching more than 5 years. (For some reason, Andy neglected to mention that; ok, probably an oversight.)

    So if we believe the Harvard Study, then TFA is significantly overstating the percentage of its alumni still teaching, since the program goes back a long time. Why would TFA do that? I can’t imagine! Perhaps they think a low number is bad?

  3. Melody:

    You’ve not responded to several past counterarguments directed at you. Why not?

    You’re also again referencing Andy and how he is spinning the data. Can you please justify this claim?

    “Here’s another take: Rotherham cites TFA stats that 32% of all 20,000 alumni are still teaching. Yet the Harvard Study that he cites finds that less than 30% continue teaching more than 5 years. (For some reason, Andy neglected to mention that; ok, probably an oversight.) ”

    Is English not your first language? Those are two very different data that you are trying to compare. TFA alumni surveys indicate that 32% of all its alumni are currently teaching. That is a different reference point than the 24% that continue to teach more than 6 years. The former measures (# TFA alums teaching a 3rd, 4th, 5th, or 6+ years) / (# all TFA alums), whereas the latter measures (# TFA alums teaching 6+ years) / (# all TFA teachers). One might naturally expect the latter to be smaller than the former. This is not evidence of “TFA significantly overstating” anything.

  4. Chris Smyr Says:
    July 27th, 2010 at 11:52 pm

    edlharris:

    If you want to ever be taken seriously on an internet forum (my instruments detect a negative on that one)

    I’d guess these are the same instruments that detect Michelle Rhee’s claim of a Baltimore Miracle is true.

    You believe Miss Rhee with her claim of success despite the lack of evidence.
    That’s similar to believing in laetrile as a cure for cancer (someone claimed it worked!)
    or in creationism.
    (Did I call you a creationist or do I think you believe in creationism?
    No, not if your claims of being a high school science teacher are correct.
    But your beliefs when it comes to Michelle Rhee are the same as a creationist.)

    (As for being taken seriously, I find the Obama administration’s idea of cutting food stamps to fund Race To The Top a sign of how desperate they are to “prove” their vision of education “reform.”)

  5. Chris,

    Earlier you wrote: “TFA as an organization is focused on the hiring, training and placement of teachers– not on political advocacy.” You then went on to mention the caveat that some alums do, but the organization’s purpose is not advocacy. By writing that, you imply that TFA is focused only on teaching– which is misleading and false. Perhaps you didn’t mean that, and then hey, I am completely wrong, and everyone can stop reading.

    Before I go on, I am a TFA alum, taught for 2 years, moved, and now work in education but not as a teacher (not in administration, and I also do not work for TFA). There, that’s my disclaimer. Disclaimers aren’t hard, and they provide a contextual basis for comments. Without getting too into it, I strongly believe your background is pertinent when arguing an issue– but that’s just me. At my school I was the “highest performing” ELA teacher when state testing data was pulled on my students– if there’s any validity in that. Anyway…

    Back to my main point, which is the advocacy bit: I think you are being a bit naive in your statement that TFA focuses only on the “hiring, placement, and training of teachers.” So they don’t focus on marketing? Branding? Expansion? Fundraising? It seems a large portion of their budget is devoted to these things.

    Or they don’t focus on the ominous sounding “Priority 3” (http://www.teachforamerica.org/about/our_growth_plan.htm), which comes right below “Priority 2,” which talks about student achievement? (If you don’t care to click the link, Priority 2 is the long-term plan to have corps members in politics, social enterprise, et c.) Oh, don’t forget “Priority 4,” which talks about the mission to “build brand awareness, and grow a still more diversified funding base.” And how will they accomplish these things? Through a large amount of initiatives (http://www.teachforamerica.org/alumni/ali/index.htm).

    Perhaps you knew all this, but just as you said it was disingenuous for whomever-that-was to say the comment about TFA needing to do advocacy, it was disingenuous– or naive– for you to stress that TFA is only focused on teachers.

  6. oh-ducation:

    Very simply put, you’re reading incorrectly into my comment. It was implied upthread that because TFA as an organization wasn’t focusing its efforts on advocating for “free early childhood education for everyone and universal healthcare”, the organization itself wasn’t doing it’s job. That’s fallacious argumentation.

    “Marketing? Branding? Expansion? Fundraising?”

    All of those are pertinent to their mission of getting good teachers into all schools. That TFA isn’t expanding and fundraising to lobby congress to pass universal healthcare isn’t indicative of a waste of money or time. A priority of TFA is to help build its alumni movement, as was already implied in my original post. That TFA is helping organize TFA alumni to enter public office but are not specifically pushing the alumni to help in their nonexistent efforts to lobby congress for universal healthcare is, again, not indicative of their failures.

    Re: your disclaimer: you are free to include all of the background information you want. You, however, are not required to do so if there isn’t any possible way you could be receiving compensation or benefits from writing here. Your teaching experience in no way bolsters the logic of your arguments, and any such appeal to authority ought not to be defended.

    Particularly due to the fact that there are commenters on this forum who would like nothing more than to alter the discourse into one of ad hominem and well poisoning, I would suggest you not give in to any possible future demands for more of your past information, as it helps the trolls divert attention from the arguments at hand. Do what you like, though.

  7. A word to the wise (and unwise):

    Do you often lose your cool while contributing to this blog? Do you engage in name-calling and putdowns? Do you reveal too much about your personal traits on this blog or your own? If so, consider using a web pen name, as many others do. Prospective employers now check the Internet regularly; therefore, real or perceived problems with communication or interpersonal relations can impede your ability to get a job, despite ability and/or education. This is especially important in fields that require good communication and social skills, such as teaching.

  8. free early childhood education for everyone and universal healthcare

    I was the first one to write the above, I think. I certainly did not say or imply anything about TFA’s lack of advocating for same (maybe Chris is just using my quote the way movie ads use their critics’ critiques). With TFA’s brilliant members, and their political clout, you’d think they would advocate for the only thing that will impact impoverished kids and neighborhoods! Oh well.

    Also, in looking at TFA’s website, I couldn’t find the “vision” referred to in their “goals.” Nor could I see where helping children was priority number 1.

    But, as Chris will point out (or not), I don’t know how to read, or something.

  9. Chris:

    You will not believe me when I say I wanted to help, so do yourself a huge favor and ask a friend or family member to critique your own blog and your responses to the above comments. Not good.

    There’s nothing wrong with your opinions, which are intelligently expressed. You are a good writer. But you need to think carefully about the way you disagree with others, especially if you are using your own name. No one wants to hire a person who deals with disagreements by name-calling and put-downs. That is a form of active aggression.

  10. I am looking up your bio, Chris. Could you at least clarify your name?

    Is it Chris Smyrniotis from San Jose? And if so, are you related to coal miner?

  11. Chris, you know, I was relatively on your side for a little bit: in my personal experience at the schools I taught, the quality of teaching was corroborated by the studies citing stronger teaching from TFA. However, then you decided to patronize me and inform me of the dangers of speaking on online message boards. Thanks for that! You’re like my dad AND mom combined! Wow!

    And, actually, contrary to you speaking for me and stating that “[my] teaching experience in no way bolsters the logic of your arguments,” it certainly does– it provides a context that I am not speaking without relevance to the issues. Subjective biases always affect arguments, especially in the social sciences– if you admit yours and are conscious of yours, you can make a better argument. Now onto the issues, because we know you like to use the words “ad hominem” a lot, and I feel I just brought this on again.

    You write, “All of those are pertinent to their mission of getting good teachers into all schools.” Really? That’s their mission? All schools, huh? Wow, I didn’t know they were placing in “all schools,” but hey, good for them. And how about that Tulsa region, huh? The one where they are basically there because they have two huge funding partners.

    Oh, here’s another thing: stop saying “fallacious,” stop writing with ridiculous amounts of double-negatives, and stop using all iterations of “poisoning the well.”

    Go on, say something about my immaturity and whatnot.

  12. Linda:

    And we finally have an appeal to fear. I was waiting for that one. I should’ve played fallacy bingo with this comment thread.

    What I find entertaining is that anonymous commenters will stop by this blog on their tour through the Internets, will sometimes submit messages that are flawed in logic and/or trollish in nature, and then will haughtily reach for further flawed argumentation when someone dares to respond. Please, feel free to hold onto your opinions, but when you post them on a public forum, expect debate, particularly if there is something incorrect about your reasoning that you may not have noticed. And if your reasoning is not only flawed but also asinine in its treatment of other peoples or groups, expect a tone of anger or jest in any replies to you. This is normally considered standard procedure for online debates, but it seems rather new to some here.

    oh-ducation:

    1) Patronize you? When did I do that? You added a disclaimer and then asserted that there’s nothing wrong with disclaimers, implying that I should do the same and talk about my past experience, and I clarified why doing such may actually work against you on a comment thread like this.

    2) You’re defending an appeal to authority and I don’t know why. Knowing that you were the “highest performing” ELA teacher says nothing about what you are current discussing in this thread. All it says is that you were likely a very good teacher, which is great, but it certainly shouldn’t make anyone more or less willing to take your arguments on faith. Context can help provide better understanding of the motives behind the argument, but it shouldn’t factor into our credibility of the claim.

    3) There’s a bit of sarcasm there, but only because you misread what I wrote:

    “All of those are pertinent to their mission of getting good teachers into all schools” doesn’t imply they are placing in all schools. It asserts that all students deserve to have a good education, and so the schools that are most likely to have a need for good teachers are the ones mainly on the list for TFA placements.

    4) And by the 4th paragraph I can see that you’re not that interested in further discussion, so that’s fine.

  13. 1) This is patronizing: “I would suggest you not give in to any possible future demands for more of your past information, as it helps the trolls divert attention from the arguments at hand.” Don’t suggest anything to me. I can make my own decisions.

    2) Next, it’s “currently.” For acting so disdainfully toward others, proofread first. Second, I never said it had anything to do with my argument– I was merely providing a disclaimer to give context to my argument, which I have maintained all along. I am a TFA alum who is not in the classroom. I fit this profile of leaving the classroom perfectly. I can admit it, and I can discuss issues from then on. I can look at TFA through a critical lens and reflect on shortcomings and successes.

    3) I didn’t misread a single thing you wrote. You said “all schools.” How in the world is that NOT SAYING “all schools?” Are you insane? YOU SAY “ALL SCHOOLS.” Stop suggesting people “misread” you when you plainly state that.

  14. Appeal to authority is a fallacy of defective induction, where it is argued that a statement is correct because the statement is made by a person or source that is commonly regarded as authoritative. The most general structure of this argument is:

    Source A says that p is true.
    Source A is authoritative.
    Therefore, p is true.

    This is a fallacy because the truth or falsity of the claim is not necessarily related to the personal qualities of the claimant, and because the premises can be true, and the conclusion false (an authoritative claim can turn out to be false). It is also known as argumentum ad verecundiam (Latin: argument to respect) or ipse dixit (Latin: he himself said it). [1]

    On the other hand, arguments from authority are an important part of informal logic. Since we cannot have expert knowledge of many subjects, we often rely on the judgments of those who do. There is no fallacy involved in simply arguing that the assertion made by an authority is true. The fallacy only arises when it is claimed or implied that the authority is infallible in principle and can hence be exempted from criticism.

    Come down off your high-horse, son. We’re being informal here.

    I posted this definition because most people don’t talk like Chris. The emphasis is mine.

    If anyone cares to look upthread, one can see the progression of Chris’s increasing disdain for dialog. It’s a shame such a bright kid is so full of himself.

    No argument anyone makes will sway his young mind. Executive Function, and all that.

    Chris, don’t be a pompous ass.

  15. Chris Smyr: ” You’ve not responded to several past counterarguments directed at you. Why not?”

    I did not respond because they were different opinions, not counterarguments. For example, I said the most plausible story for survey non-response is X, though there may be others. You said Y is more plausible. That’s fine, neither one of us has evidence to back up our opinions, so we disagree.

    I said I thought results from a survey with only 60% response were unreliable. Someone else said, no 60% is fine. That’s not a counterargument, it’s an opinion. And it depends. If you don’t care too much about accuracy, then I would agree that 60% is fine. One could get into the statistical woods and argue about Manski bounds or Heckman corrections or whatever, but this doesn’t seem like the place for it.

    “Those are two very different data that you are trying to compare. TFA alumni surveys indicate that 32% of all its alumni are currently teaching. That is a different reference point than the 24% that continue to teach more than 6 years. ”

    Thank you. That’s a very good point. I poked around on the TFA website, and it seems like the size of TFA cohorts has grown considerably over time. So the ranks of all TFA alumni is heavily weighted with more recent members, who would of course be more likely to be teaching than older members. That makes Rotherham’s citation of the figure as evidence of large numbers staying in the classroom longterm even more questionable. He is truly to be commended for his nice spin. Thank you for helping me to make the point.

  16. oh-ducation:

    1) Not all suggestions imply a patronizing tone. The suggestion wasn’t for your own good, but rather so that we can have discussions here without the need for further background checks from anonymous experts.

    2) Here is what you said:

    “And, actually, contrary to you speaking for me and stating that “[my] teaching experience in no way bolsters the logic of your arguments,” it certainly does– it provides a context that I am not speaking without relevance to the issues. ”

    You implied that an added context has bearing on the logic of your arguments. All I said was that it doesn’t.

    3) You’re a bit hard-headed on a relatively minor point:

    “All of those are pertinent to their mission of getting good teachers into all schools.”

    I am not implying TFA is putting teachers into all schools (obviously), but that the mission is to make sure good teachers are spread among all schools. To do so requires TFA to work toward getting their teachers into schools that have the most need, until one day, good teachers will be in all schools. Not all “good teachers” have to be “TFA teachers”, and thus there are many schools that compose a part of “all schools” that TFA need not place in.

    TFT:

    You seem to be confusing a common definition of informal with another, more nuanced definition, and so I’d suggest you read up a bit on informal logic.

    Not only that, but your bolded paragraph from Wikipedia already offers a response. If one chooses to argue from authority, the authority first ought to be related to the arguments at hand (being a teacher does not mean one is consequently an authority on education research, or for that matter on TFA hierarchy), and furthermore the appeal to authority does not logically stand up to criticism and further argumentation. Being an authority on an internet forum means very little, since as soon as a counterargument is raised, that authority figure must defend his/her claims, or else lose the debate. This holds true for informal logic, as well.

  17. Melody:

    1) The fact is, you provided an alternative hypothesis that requires a key premise (“TFA teachers who leave their school after their 2-year commitment do not feel proud enough to respond to surveys about their experience”) you’ve neglected to substantiate. The actual response rate further argues against your premise, in that it mirrors the typical response rate seen by TFA teachers on similar surveys. Please don’t try to relegate this as a difference of opinion when it is really more than that.

    2) Your argument has spiraled into all sorts of tangents, but alright:

    Andy had referenced the study we are referring to in response to the claim by another study that “more than 50% of TFA teachers leave after two years.” Either number originally advanced by Andy in reply to this original statistic (from Harvard or TFA), due to the methodology of the study, gives a better bearing on the retention rates of TFA teachers than that claim quoted above. Neither of these two values is evidence of anyone spinning the data; rather, the two different numbers are simply two different data that can separately inform your arguments.

    TFT:

    Neither are you. We’d be tied at this fun game were it not for the fact that you’ve offered very little in terms of cogent arguments for the past 60+ comments.

  18. The author writes:
    “…the Berliner study wants for rigorous methods: Before you even get to the statistical sleight of hand, which isn’t that complicated to ferret out, the selection problems undermine its methods.”

    But then the others holds up for praise a study of TFA retention rates based on a survey of TFA alumni. It is likely that that those alumni willing to participate in a study of TFA alumni are more likely to be teachers than those who left teaching behind after a few months or a couple of years, and now (years later) are uninterested in participating in a survey about their experience as TFA’ers.

  19. Trulee Pist:

    It’s not necessarily so that older alums will be less willing to respond than younger alums. Even those who are still teaching will not necessarily respond more frequently than those who are not teaching. Upon completing the two-year commitment, directors will meet with TFA corps members and determine if they will be staying or going, and that at least will provide an assessment of nearly 100% of TFA and determine if they are being retained after 2 years.

    Were one to still find the TFA alumni survey results hard to believe, one could just look at the Harvard study and find similar results for retention.

  20. Why should districts spend $1600 per TFA teacher? When you think that generally they have to put out this amount every two years multiplied by how ever many TFA teachers they hire….I don’t see where the slight benefit in math scores is worth it. The study showed they didn’t raise reading scores. Also, this study was done when uncredentialed teachers were allowed to teach. That is not the case anymore. Now you must be ‘highly qualified’.

    Also, nice that they get loan forgiveness while many traditional teachers in the same schools don’t. No wonder it’s so attractive, especially in this economy…plus a free masters? Hmmmm.

  21. To Butternut-
    A large study done by Stanford showed Charter schools to be 83% as good or worse than public schools. They aren’t as great as everyone claims. Also, many people don’t realize that charters can cherry pick kids, kick them out or pressure them out if they are not up to their standards. PLUS…they have lots more money that actually goes into the schools. Things like technology, materials, field trips etc. There is a vast inequity between what you find in inner city and suburban or charters.

    And if you have read anything about KIPP….well I guess if you want to go back to sweatshops. They make teachers work Saturdays, evenings and be on call. I am sure if everyone spent that much extra time at their job, their productivity would go up. But that is why unions were formed for fair working conditions. KIPP teacher turnover is high due to that. Is that really the best system? I don’t think so.

  22. A couple of comments from a newbie here,

    I appreciate Chris’s sticking to his guns on logic.
    If it’s only about logic, *his* background is not relevant to his arguments, no?

    I also appreciate others’ willingness to weigh in with opinion.
    Their backgrounds seem more relevant.

    Anyway, I have a couple questions, and maybe some opinions as well:

    The studies showing TFA practitioners as equal to or better than traditionally prepared teachers (TPTs)- Were they measuring against teachers in the same schools, or against all teachers? I guess this is relevant in regard to the suggestions in this thread that better teachers are somewhat more likely to be at schools where TFA does not go.

    Retention rates- I haven’t seen rates for TPTs in this thread, but I understand quite a few leave the profession in the first 5 years. Does someone out there have access to those studies? It would be interesting to compare TFAs to TPTs, instead of just citing low retention as a negative for TFAs.

    Now to opinions:
    First, my background- I was recently laid off from my avocation as a teacher (14 years) at a Title One school. As a 55 year-old, I have to be open to the possibility that I will not be able to practice my profession again (at least, not as a paid professional). I am not bitter about the decisions my impoverished district had to make, but I am very concerned about the tone of the national discourse on education.

    So…
    TFA is a bandaid. I say this because I imagine (no studies or other documentation here) that it is not scalable to the point where it could replace TPT. It is an elegant bandaid and may be an effective one (not sure yet). But, as a bandaid, it is an enabler.

    It enables us to continue to shortchange schools in poorer areas by diverting the discussion away from resources. Yes, teacher quality is important, but resources play a role in determining the allocation of quality teachers. Those districts with the resources can afford to be extremely picky in their hiring and generous in their retention strategies. Poorer schools resort to only hiring BA first year teachers to keep the salaries down and then mix a sigh of relief with regret when those teachers go on to better pay, better working conditions or better prepared students at other schools (you know, you can go from being an abject failure as a teacher to a rock star, just by changing the zip code of your place of employment).

    It also enables us to sidestep another issue, how to make teachers better at what they do. We believe that all students can learn, that the right teacher can make a difference in the life of a student. Why don’t we apply that to teachers? The existence of and discussion (sometimes endless) about TFA allows us to continue to ignore the development or even rehabilitation of the teachers we have. It’s simple. You expensive teachers can be replaced by cheaper ones with (sometimes) equal or better results.

    TFA is not, of itself, bad. We should be able to learn from it. What makes the successes happen. What are the downsides? How can TPT programs use the info to be better? How could TFA learn from successes at TPTs?

    It seems that TFA works from the premise that expertise is important, while TPTs stress the importance of preparation to teach. The New York Times (certainly not a peer-reviewed journal) ran an article showing stories, narratives, where each of these premises bore substantial fruit. http://www.nytimes.com/2010/03/07/magazine/07Teachers-t.html

    Enjoy.

  23. Southeastern state: 90% of TFA teachers no longer employed as teachers in the state public schools in year four of beginning teacher cohort.

    Report will be released this fall.

  24. Nava:

    1) School districts are not at all required to spend ~$1500 to help fund TFA’s recruiting costs for each teacher, as you seem to imply, but many find the benefits of gaining such teachers outweigh the costs, which are, relatively speaking, quite cheap.

    2) Loan forgiveness? A free masters? What is this special TFA program you speak of and why didn’t I join that corps instead….

    3) “[KIPP schools] make teachers work Saturdays, evenings and be on call. I am sure if everyone spent that much extra time at their job, their productivity would go up.”

    It shouldn’t be considered “extra time” if they are getting extra pay, as they receive 15-20% higher salaries than the average teacher in neighboring schools.

    Artie:

    The Mathematica study used random assignment and control teachers within the same schools and grades. Also, the retention rates seem similar between the two groups if you look at TPTs in high poverty schools (where TFA primarily places corps members). One of the studies TFA has on their page cites “No Dream Denied” (http://www.nctaf.org/documents/no-dream-denied_summary_report.pdf), which summarizes the data at that time with the following (and a subsequent plot): “The turnover rate for teachers in high poverty schools (~20%) is almost a third higher than the rate for all teachers in all schools (~15.7%)” After 2 years one might expect a ~36-40% turnover in high poverty schools, which is a close match to Harvard’s TFA study of 39%.

    I disagree about the band-aid claim, or at the very least when it is argued with negative connotations. Just as the achievement gap persists for multiple reasons, so similarly plentiful are the needs of public education that have to be filled to close these gaps. Getting good teachers into places that need good teachers is one logical way to help close these gaps. Making teaching a more valuable option to future leaders from top-tier universities is another way to help close these gaps. Empowering future leaders with a burning desire to advocate for their students’ successes in many different arms of society is yet another way. Teach For America seeks to fix what is broken on a number of different levels, and to attack it for what it *doesn’t* do seems to be unfair criticism.

    TFA is all about these necessary discussions you’re addressing, and it’s the hope that with the alumni movement that they are building, they will become more and more numerous and in the right places. But while this fight is being fought outside the classrooms, what else can we immediately do inside the classrooms other than provide better teachers where they’re needed?

    I also think that TFA would offer a valuable case study for learning how to “rehabilitate teachers”, to make sure the best are retained and to shoot for continual improvement. TFA has a strong focus on accountability and working to improve day-to-day instruction, while maintaining the flexibility to respond to insight and feedback offered by teachers to provide its corps better support, and more of that could be useful for all teachers.

  25. Chris,
    Thanks for the link (it didn’t work as clicked, but I found the document you referenced). Interesting work there. I”ll need to go over it some more off line to get it all.

    Two quick points:

    1) The summary differentiated between turnover and attrition. The beginning teacher loss was given as attrition and, if you extrapolate from the turnover rates showing a 1/3 higher rates among teachers from high poverty schools, using the attrition part of the graph, it comes to about 30-33% for high poverty teachers, depending on whether you compare them to all teachers or public school teachers (It’s a small difference, I know). Interesting to note that private school teachers have a generally higher attrition rate than their high poverty public school colleagues.

    2) Sorry if this sounds a bit picky, but when you mentioned making teaching more valuable to future leaders from top-tier universities, I hope we can include some future leaders from other schools as well. And perhaps we could make teaching itself more valuable (as suggested by the study you referenced), so those of us who wish to stay in the classroom don’t have to second guess ourselves about not becoming administrators for higher salaries.

    Thanks again for the response. I’m holding on to the band-aid metaphor for now, but allowing that band-aids are useful if they are part of a larger treatment of underlying causes.

  26. Chris, the Evidence on TFA-effects are mixed at best. Given the high cost of teacher turnover, it would seem wise for a district to try to hire a well-qualified teacher who just might stay 5-10 years rather than someone you know will be out the door in 3.

    Someone should do a study on the messages sent to poor/minority kids when teachers turnover at such a high rate. A dissertation found that the kids internalized the belief that no one really cared about them when teachers turned over constantly like TFA teachers.

    Don;t believe the TFA study on retention. Fatally flawed. It would thoroughly discredited by you and Andy if it showed something you did not believe was true.

  27. Billy Bob:

    1) As Andy points out, the studies with sound methodologies tend to show TFA teachers are just as good as other teachers.

    2) Given that TFA places in schools where the school usually can either, A) hire a TFA teacher, or B) hire someone else with an equal amount of teaching experience (likely none), your suggestion doesn’t seem that helpful. Also, the “well-qualified” term is a rather loaded one for this discussion.

    3) TFA teachers “just might stay 5-10 years”, as well. A quarter of them do, at least.

    4) Can you cite that study please? I don’t see how poor/minority kids should be affected if, say, by their 11th grade year their 9th grade teacher decides to leave.

    5) Can you substantiate your claim for why the TFA study is “fatally flawed”?

  28. Sound methodologies according to whom? TFA. C’mon.

    Look at the sample sizes of the Mathematica and NC papers–Very, very small indeed. No one mentioned that, did they?

    Dissertation is not yet complete. Will be finished this month.

    TFA study is fatally flawed because of low response rate and no evidence that bias was not introduced due to low response rate. Guess you have not taken research courses yet.

    Istead of relying on TEA, we could actually get great people to go and stay at such schools if we took the money from what is going into charter schools, TFA, and other such boutique programs for white people who want to feel they are helping poor kids before they make their millions and pour it into getting good principals and good teachers at schools with wrap-around services.

  29. Billy Bob:

    1) Nowhere has TFA itself judged the methodologies of these studies, nor has anyone here defended a study because ‘TFA said so’.

    2) What scale are you using to determine that a sample size of ~2000 students spread out over 100 classrooms–or 2029 teachers in the Harvard study– is small? The former even utilized random assignment to help further validate their sampling.

    3) Since the dissertation hasn’t been finished yet, nor published, nor peer-reviewed, maybe you shouldn’t yet cite it as evidence.

    4) You (and others) have not yet been able to substantiate this “62% response rate = bad” claim, nor has it been reasonably justified why it is that we should expect that this sampling invalidates the results. Maybe you ought to do this first before making faulty assumptions of my background.

    5) That last paragraph would make a great mission statement for your new teacher prep program. I’m looking forward to hearing about all of your future successes.

  30. Chris–you obviously did not go to the link provided in the post that reviews the studies. There on the TFA page is a ranking of the methodologies. The quality of the study is inversely related to the positive nature of the findings. Shocking I tell you! Shocking!!!

    Some studies cited by TFA folks have small sample sizes. I know the NC one does. Ill have to re-examine the first mathematics study.

    Dissertation will pass.

    Dude–general research methodology says 62% aint so good and especially if you cannot validate that the non-responders were no different than responders. Which TFA cant do. “nuff said. And, at least I finished my PhD, unlike some ed reform bloggers.

    Working o it while stopping the funding of TFA and KIPP.

  31. Billy Bob:

    1) Reread what I said: there hasn’t been a ranking of study methodologies with TFA doing the ranking. The figure you are referring to is a visual display for the Education Next Report Card. If you read more closely, it’s also not ordered 100% inversely to the positive nature of the findings.

    2) Cool, so get back to me when you re-examine the Mathematica study, but maybe you should do so before you assert that studies like that one are invalid.

    3) Still haven’t given any justification as to why we’d expect non-respondents to offer different answers than respondents. A sample of 2000 teachers should be large enough to make the results meaningful.

    4) That you have clarified that your “dissertation will pass” doesn’t mean it’s alright to cite it as evidence. You probably should keep your ego in check until at least it’s published so the rest of the academic community can then judge it.

  32. Its all the same people–whats the difference? Its all group-think, band-wagon people giving a reach-around to each other.

    You clearly don’t understand research. It is incumbent to examine potential bias when response rates are low. Those not staying in education could definitely be more likely to not respond given the whole ego trip TFA people are on.

    The dissertation has gone through more peer-review than most of the trash that Andy cites as evidence on this blog like the crap that public impact puts out or the UVA turnaround people. Worst evaluation ever.

    When you finish your PhD and do some real research of your own, come back and we can have a discussion about research, Until then, you don’t even know that you don’t know what you are talking about.

  33. Billy Bob:

    1) Thanks for clarifying that Education Next is just TFA in disguise. I’ll write that down so I don’t forget. Of course, none of these ratings and reviews have relied on appeals to authority as all have given evidence in their critique of the methodologies, but that surely doesn’t matter.

    2) Great, now try examining that potential bias and offer justification for your assertions. You are implying you are the expert on research here, but your arguments (and lack thereof) tend to disagree with that sentiment.

    3) So your dissertation has gone through more peer-review than, say, the Mathematica study? Even though it isn’t finished?

    4) When you get that dissertation published, we can talk more about what “real research” looks like. In the meantime, you ought to work on the critical thinking skills that should accompany that PhD.

  34. I can definately agree with Linda/RetiredTeacher were she Says:

    July 27th, 2010 at 12:04 pm
    Here’s something we can all (hopefully) agree with:

    Let’s end the practice of placing the least experienced and least prepared teachers in our most challenging schools.

  35. ED Sector and the Problem of Study Laundering:

    EdSector CMO Report: Who Lost Tom Toch?

    Thanks to a couple of eagle-eyed readers (including MDM) for pointing out that the much-delayed Education Sector report on charter management organizations lacks the name — and apparently much of the content provided by — its original author, writer and EdSector co-founder Tom Toch.

    Asked about the situation, Toch said, “I removed my name from the report because a good deal of my analysis was removed and, as published, the report does not reflect my research findings on the current status and future prospects of charter management organizations.”

    Toch says he submitted the 20,000-word report in June, based on two years of investigation, but did not see anything further until a paper copy of the final report was shown to him this past weekend. A good deal of the candid commentary from those within the charter community “had been removed,” according to Toch. And the report recommendations were added on by someone else.

    Toch can’t publish the original version of the report because of copyright issues but he points to several other pieces (in Education Week and the Kappan) that reflect his findings more completely, and notes that he will continue to write and speak on the issue.

  36. PhillipMarlowe:

    This might be a good time to share this information with the New York Times and other major newspapers. Thanks for sharing.

  37. Another measurement of TFA success would be how well did the students they teach do on AP tests.
    What percentage of students taught by TFA received a 4 or 5 on their AP Biology, AP Latin, AP French test etc, compared to the non TFA teacher.
    This way we can compare a test with more value than what is used for NCLB.

Leave a Reply

Your email address will not be published.