Friday, June 09, 2006
The Smoke Clears...
AFTie John has just realized that the Democratic governor in Iowa signed an education tax credit bill; the one in AZ did the same thing a few weeks ago. Yeah, we're getting our asses kicked. Stop reading Boardbuzz and start reading the newspaper! These bills are not great policy but this is what happens when, as President Clinton puts it, you try to beat something with nothing. Democrats need something to say on this issue of choice that isn't just "no" and is not education tax credits (which are really lousy policy) and vouchers (which have their own set of problems). For some reason that's politically hard right now...maybe AFTie John has some thoughts on why? Update: Andy Smarick says it more eloquently than I did.
Also, AFTie One-L protests that per the post below in fact she doesn't hate the Ed Trust report, just some of the recommendations*. To the more central point she's touting, the AFTie data in question isn't as definitive as One-L likes to think because (a) it's too aggregated, meaning district size is a big variable here (b) the distinction between states with collective bargaining and states without it is not a clean one at all in terms of practice at the local level (c) more cumulative data would be more useful and (d) we have no idea about what rate of turnover is, for lack of a better word, "ideal." We know that a zero percent turnover rate isn't good and that 25 percent seems on its face to be too much. But, because issues of human resources and talent in education has received so little attention it's hard to really get at this issue the way you can in other fields. For instance a big accounting, consulting, or law firm can tell you with some certainty where they want to land on this question to maintain the talent pool they seek. Those questions are not, pace One-L, an aversion to data rather it's the common sense step of making sure it's applicable to answering the question at hand.
*Update: AFTie One-L is trying to shift from cheerleader to referee and now says "I did not say that [AFT researcher Howard] Nelson's work is the definitive study on teacher transfers--I said that he took a look at the data and what he came up with ran counter to the New Teacher Project's (NTP's) study." But what she actually said was "I guess no one from Ed Trust attended the Ed Sector event where AFT researcher Howard Nelson convincingly deflated the argument that teacher transfer provisions are the reason urban schools have fewer qualified teachers. "Convincingly deflated" sounds pretty definitive to me, no? Anyway, you can decide about the debate yourself, the transcript and audio are now online. Also, here is the NTP study in question, read it yourself. Not definitive but more granular. Would be nice to see more data like this from more places.
Update II: It's Hard Out Here For An AFTie! Now One-L is again showing worrisome signs of relativism in how she argues: "Did I find Nelson's data analysis as convincing as Eduwonk found the New Teacher Project report? Yes." If I said that I didn't think the NTP report was convincing would she attack poor Nelson? C'mon.
Anyway, problem for her is, I never said I was a referee here, that was her posture and that's why we had a journalist moderate the debate in question rather than me (and yes, yes, the AFTies give a platform to different views all the time…). Thing is, none of this is definitive (though I think the evidence points to an issue), and that's why Jane Hannaway and I called for a lot more research and analysis in Collective Bargaining in Education, but it's easier for me to say that because I don't have a party line to toe. I'd really like to see a lot more research around these questions. A joint NTP - AFT project to look at a bunch of districts would be a great place to start. Right now the NTP method is superior to the AFT one so if the AFTie gripe is that NTP only looked at five districts, then let's expand the analysis.
If you follow or are interested in the high school issue don't miss this outstanding new paper by Aspen Institute Senior Fellow Judy Wurtzel (pdf). The entire thing is well worth reading and offers an important analytic framework but the discussion on professionalism is not to be missed. Aspen has been getting analysts and practitioners together to work through these issues and you'll see some of the various new ideas reflected in the analysis as well as some more established ideas. For good background on recent high school efforts to date and some dimensions of the issue you can't beat this ES report by Craig Jerald.
Also, new Education Trust report on teacher quality. Teaching Inequality: How Poor and Minority Students Are Shortchanged on Teacher Quality (pdf) is aptly titled. A lot of data and sensible recommendations. AFTie One-L responds here, she hates it! Q and E loves it! Not sure though the contract issue is as settled as One-L claims (though she obviously subscribes to the "if you say it long and loud enough it becomes true" school of thought). You can decide for yourself soon when ES posts the audio and transcript from the debate. But the new AFTie data (pdf) is (a) too aggregated right now to really be useful and (b) starts from an assumption that less churn is always better. I think we can all agree that too much is bad but the question of where the "sweet spot" is more complicated and the aversion to using data in education makes it a tough one to answer in the way that other industries can.
Also see this statement on the Ed Trust report from George Miller, the ranking Democrat on the House education committee (Also, USS Barack Obama here). Sure seems like there is an opportunity here for Democrats to (a) legitimately hit the Bush Administration on failing on this front (b) stand up for a constituency they ostensibly care about, poor and minority kids and (c) put forward good solution-oriented ideas. Of course doing so means sticking more than a toe into this producer-consumer issue and everyone knows that the producers are so great on all those progressive issues that it's just irresponsible to cross them!
Another Blow...
Man, the NEA's legislative goals are just getting beaten back daily...
Also, related, if you're a conservative Republican Hill staffer, getting worried that you might lose your job in November despite Tuesday's allegedly bellwether CA-50 special election, then this job at the NEA might be just for you! Hang out with your old pals in the Republican Study Committee, work on all the shared goals, and make more money! What's not to like?
Despite some questionable aspects of the policy, I was surprised by the margin of defeat for the pre-K initiative in California. Sara Mead explains what happened and why this might actually be good news for pre-K advocates in the long run.
My colleague Tom Toch makes the point that regardless of whose estimates you believe in the ongoing graduation rate debate they still point to a substantial problem and many of the preventative steps would be the same in either case. But Tom then goes on to say that the research dispute about methods doesn't really matter that much at all. There are some arcane disputes in this business, but I'm not sure that latter point is right in this instance.
I do think the grad rate debate is pretty one-sided and mostly political at its core but it is really about different datasets rather than methods (in other words the methods conform to the competing datasets). I don't buy that the EPIers data is superior and while the various cumulative methods have some shortcomings I think they're the best at hand right now for information and current estimates.
But, that said, if the EPIers are right about the datasets it still has pretty big implications for public policy in ways other than the general issue of combating the problem. That's because the National Governors Association and almost all the nation's governors have embraced a compact (pdf) essentially based on the Greene method, Ed Week is about to launch a new project to report grad rates based on Chris Swanson's method (he now works at Ed Week) so that's going to become part of the conversational grammar, and these numbers have real consequences for state accountability systems and No Child Left Behind going forward. So while the general policy prescriptions might remain the same in terms of dropout prevention, these numbers do matter in terms of holding schools, school districts, and states accountable in practice and with the public. Consequently, seems to me that whether they are right, or as right as can be, does matter does matter at that more micro level. It's a distinction with a difference.
Behind the scenes National Education Association apparatchiks and FairTest activists are furiously waging a whispering campaign to convince people that though the Connecticut NAACP intervened in the anti-No Child Left Behind lawsuit on the side of the Bush Administration neither they nor the national NAACP really support the law at all and are in fact right there with NEA-FairTest. Problem is the actual filings (pdf), press releases, the statements of NAACP officials in CT, and other actions show that the NEA-FairTest line is basically, well, propaganda. The national NAACP is on board with the latest NEA-ginned up sign-on letter calling for changes to No Child but they signed-off on the decision for the CT chapter to intervene (courageously it should be added because the NEA really went nuts and can bring a lot of pressure to bear on this kind of stuff). One of these things is politics, sign-on letters don't change things and it's easier to go on than to stay off. The other action is actual litigation, that does change things. Also, related, is this op-ed from earlier in the week by California Secretary of Education (and ES board member) Alan Bersin and California NAACP President Alice Huffman. All this shows the extent to which the NEA and FairTest are fighting a rearguard action in terms of addressing the caffeine headache. But give the NEA folks some credit, while their buddies at the Republican Study Committee are trying to build a bridge to the early 19th Century, the NEA is only trying to build one to the 1980s.
Important new paper from the National Charter School Research Project. Lays out some conceptual issues for charter school research (applicable to school choice research more generally) and is a useful metric to apply various studies against. The project, directed by Robin Lake (at left), hosted a valuable, very well-attended (for a private meeting), and ecumenical meeting last Wednesday at Brookings to discuss and release the paper (Ed Week's Usually Reliable Robelen reports here). For my money there are a couple of issues:First, while studies vary in their quality and generalizablility, that worries me less than how they are used. I still think it's good that charter schools be included in the NAEP, for instance, even though that data is then inappropriately used in some cases. I actually subscribe to the "land of the blind, one eyed person"(that's the PC version) school of thought and even crude descriptive data is better than nothing if interpreted with appropriate caveats (and even more rigorous studies need to be interpreted with the appropriate caveats and in context). But, charter politics being what it is data are often misused to score political points.
Second, charters and chartering varies so much nationally and increasingly even within states, that studies are going to have to become more granular. As a matter of public policy generalized studies about charters have some utility though less as the data becomes more aggregated. But, in terms of learning from/about specific schools or school types they are of limited use. If NCLB causes a lot of charter conversions this issue will become even more pronounce as there will be more schools called charters that have little in common except that they were lousy schools before they became charters.
Third, I'm not sure how much all this matters, unfortunately. There are folks who hate charters, and will continue to, regardless of the data. At its core this is a debate about power; who should have it and be able to wield it in American education. Conversely, some charter supporters just don't like the folks who have the power today, teachers' unions and school districts and so forth, and they're going to support as much choice as possible regardless of the data. In other words, a lot of this is politics.
But, more choice and pluralism in public education is here to stay. So, in the end, research like this is important to inform the policy process so when the pressure does build-up one way or the other and policymakers turn their attention to this issue there is some empirical work for them to draw on. That's why the NCSRP is an important undertaking and a project well worth following. Disc: I'm on the advisory board.
The NEA's Goals
Per this item below about the NEA's goals, here's a handy 10-point list you can carry in your pocket if you're on the NEA dole and need talking points. Can't wait to see the next candidate questionnaire...Update: Bummer, one of the NEA's goals was soundly defeated in the Senate yesterday. But buck-up guys! Still plenty of time to curtail a woman's right to choose, it's only June!
This is an interesting entrepreneurial idea but there is so much free content out there (and even on this site) I'm not sure how the business model works over time except in the Googly way of using it to leverage other revenue streams.
Really. Update: This picture of the alluring Ms. Craig accompanying her press release has prompted several
Serious Education Journalist Alexander Russo is fast becoming the Joseph McCarthy meets Willy Loman of education blogging. Today's post on the Widmeyer Ed Week commentary (see this item below) is an archetype of his method: Vague criticisms with no specifics "To my reading, the advice includes a lot of self-contradictory PR mumbo-jumbo and some notable clunkers"; a passing smear, in this case a link to a conspiracy-oriented article in The Nation that took a pop at Widmeyer without noting his history on the reading issue; and then a thinly veiled pitch for freelance work, in this case an article about Widmeyer.
Art Wise takes the wind out of the sails of NCATE's critics? Backstory here.
Sam Freedman on collegiate grades, interesting column.
In an Ed Week commentary Scott Widmeyer writes-up a sensible list of guidelines for education communications, well worth reading.
Meanwhile, Michelle Davis reports on the rise of the Republican Study Committee comprised of the serious House conservatives and the eduimplications but seriously buries the lede at the very bottom. Davis:
Randall J. Moody, the manager of federal policy and politics for the 2.7 million-member National Education Association, which is typically allied with liberal House members, said he could envision a scenario in which his Washington-based teachers’ union works with members of the RSC.
“Obviously, they have a philosophy which should in some ways mesh with ours in terms of empowering school districts and teachers at the local level,” he said. “In reauthorization, that coalition might be helpful.”
He acknowledged that “in terms of money, they haven’t been helpful.” While the NEA has repeatedly called for more federal education funding, the RSC has worked to cut the amount of the federal budget spent on education. But that doesn’t preclude working together, Mr. Moody said.
Though he would only say “possibly” when asked if the NEA has already been in contact with RSC members about the law’s reauthorization, he emphasized that “our goal here isn’t partisan or ideological. ...We work with those who share our goals.”
Share their goals? The Republican Study Committee romances localism and has a goal of building a bridge to the early part of the 19th Century...Sure, in terms of money they "haven't been helpful," but on exactly which Democratic/liberal/progressive issue have they been helpful? The NEA is, we're told, a key part of the last vanguard of progressivism in our country. Please spare me that nonsense. They'd happily work with the RSC to eviscerate the provisions in No Child Left Behind that are squarely aimed at leveling the playing field for poor and minority youngsters. For the RSC it's about federalism and at least that's a real and intellectually defensible principle. What's the NEA's excuse? File this one squarely under with friends like these Democrats don't need Republicans. Davis caught the NEA in the act and it again shows they will stop at nothing to undo this law. Why did Ed Week bury it at the end of the story? See also this topical essay by UFT's Leo Casey on this issue.
In an interesting post about symbolic politics Sherman Dorn chides me for having the right attitude about the NAACP's intervention in the CT No Child case but then ignoring that the state wants to keep test quality high something that he notes I've written about previously and that my colleague Tom Toch wrote about recently in this ES report.
In theory Sherman's right that there might be a contradiction there but I don't think that CT's case is going to hold up on the specifics. First, they've been quite slippery about what exactly they've been spending money on and the case is not as strong on the money as some are claiming. Ultimately, though, that's for the courts to decide. Second, there is a rhetorical trap here where multiple-choice testing becomes axiomatically equated with low-quality testing. Leave aside basic issues like curricular alignment, there are high-and low-quality multiple choice tests. Conversely, just because a test has open-ended or constructed response questions doesn't guarantee that it is a high quality test for the purposes that it is intended for or overall. In fact, there are some complicated issues here in terms of test reliability that have legal implications for high-stakes testing with non-multiple choice tests. As a matter of policy and practice, for my money annual testing can and in fact should be done with a combination of different kinds of tests to minimize the time students spend on testing but maximize the data educators and policymakers need.
This all matters in the case of CT because the argument that CT cannot do high quality tests without more money from the feds is far from a slam dunk. It's entirely possible that the state can do just fine on the quality front but that they want to do different kinds of testing. That's a legitimate desire and though I've argued for a long time that we should be spending more on assessment from the federal level than we do now, desire is likely not controlling in this case rather a minimum floor, laid out in the statute itself, is. In other words, Secretary Spellings may have been overly dismissive of CT's concerns but she's probably right on the substance and will probably win in the end (though perhaps not as quickly as in the Michigan case because the issues are more complex). The disconnect here speaks to the need for a centrist coalition that supports testing but wants high quality testing. The still not-too-hidden testing good v. testing bad debate hinders progress on this front and of course the weird politics about the federal role which transcend left and right only further complicate matters. In the end though the CT case was/is mostly about politics not substance so again go read Sherman's apt post! Also, while you're there check out this interesting post about Sherman's time in the pokey.
Rebranding
Over at the AFTie blog One-L is again doing something that's not gathering signatures. This time it's attempting to re-brand AFTie John by invoking a great musician. But regular Eduwonk readers are not fooled and would recognize AFTie John anywhere!
I'm sure this is a terrible idea that is ruining childhood but I can't quite figure out why, seems like a good idea.
Interesting post from a UFT member complaining about the new contract. Two takeaways: First, not as easy to be a reformer as my friend Leo Casey says. And, per the post itself, in what other industry do you get to walk in and have a claim on a job? Not saying that seniority shouldn't count for something but they days of, "you pick the school you like and simply go there" are thankfully increasingly a thing of the past. Update: The UFT member writes to say: I'd like to clarify something -- your point about which industry allows you to do this stuff is certainly valid. However, the UFT transfer plan was given to us in exchange for one, or two zero percent raises. It's simply bad business to give up something for nothing. Also, however you may feel about this or that plan, it's disingenuous and outright ridiculous that Mr. Mendel present this change as a UFT victory.
This is a pretty cool site from CA that rolls up a lot of data and information in one place. It's the result of the CA Ed Data Partnership. Good model.
Oh brother. NYT's Gootman turns in a must-read. Joe Williams has been following this for a while, click back through the links for the whole saga. Here's the interesting thing about Joel Klein: He's the head of a big school system yet he wants to give more power to parents and others at the expense of the system by supporting more charter schools and in the process he's antagonizing parents who should be his (and his boss the mayor's) natural power base in an effort to help the parents who perpetually get the least. Education politics are fundamentally power politics and this basically cuts against the grain.
From the NYT:
But after his election to the city's top office last month, law enforcement officials say Mr. Booker now has to endure one of the grimmer aspects of the political experience: a death threat that the police are taking so seriously that they have expanded his personal security team.
is moving forward...background here, most recent update here (pdf), press coverage here and here. Seems like a good idea for some of the '08 wannabes from both parties (or even this party) to champion. Disc: I'm on the advisory board.
Teacher Matt Matera's students are putting on Midsummer Night's Dream and are blogging all about it. Check it out.
I've been meaning to write about supplemental services (publicly funded tutoring under No Child Left Behind) in the wake of the Secretary's letter to the chiefs a few weeks back but haven't had time. In sum, that whole policy needs a lot of work from the fundamentals underpinning the initiative through the implementation and I still don't think the administration gets it.
But, that said, today's Washington Post has a big splash about how rural school districts cannot implement the provisions. Now rural districts (and rural communities more generally) do face some unique challenges as a result of size (or lack thereof) and geographic isolation. And there can be little doubt that brick and mortar tutoring companies are not going to locate in all rural areas. But when Montana tells the WaPo that only 20 of 14,000 eligible students at 66 schools have received tutoring because of these issues, the BS detectors should go off and some hard questions should be asked about how vigorously the state and school districts are implementing the provisions and genuinely trying to serve those kids brick and mortar or otherwise. And on the otherwise, while I'm certainly not a technotopia type and think that a lot of ed tech is over-hyped, the WaPo also has a story today about how DC-area Somalis keep up with goings on back home in real time using technology, it's an interesting juxtaposition with the attitude of the rural states/school districts that there is just no way to do this stuff...not saying we're going to deliver tutoring over cellphones, just that there is more play there than 20 of 14,000 if the state really wanted to see those kids served. Readers are left wondering how hard WaPo's Goldstein pushed them on that point...
Also, good a time as any to note that there was a lot of bloggy grumbling about the appropriateness of former PA state chief and Bush Admin. Deputy Secretary of Education and current lobbyist/advocate for supplemental services Gene Hickok's WaPo op-ed about the tutoring provisions because he obviously has skin in the game. I don't carry any brief for Hickok but I don't think the Post did anything wrong by publishing the piece since they disclosed that he is a player. In the ideas debate sometimes people who are players have insights worth sharing (sometimes more informed insights than just observers) and disclosure is the key ingredient. But, it would have been good for the disclosure to note that WaPo Company property Kaplan is a player here, too, and is involved with a key Hickok client. The disc did fall short on that front.
This is for the good, despite all their problems DC's public schools have remained pretty hidebound. Speaking more broadly, little teacher pay pilots like this are springing up more and more. It's a typical response to pressure for change: Make a small change at the margins, incorporate the reform, move on. The risk is that rather than be the beginning of broader changes, these small pilots become Potemkin PR tools to argue that "this is being done, nothing to see here" and media and political elites go for that in education a lot and do move on. It's one reason the "system" is so resilient. Almost everything is being done somewhere, but some things need to happen almost everywhere.
It's On Baby: Carey V. Wisconsin
Speaking of utopias, a Wisconsin librarian says Wisconsin is one on education, Kevin Carey says it's "mediocre at best." You decide.
Eduwonk reader and Columbia TC student Jennifer Booher-Jennings has an article in the new PDK about "rationing" education in an accountability driven system. Worth reading but her second dilemma seems a more real tension than the first. That's because the percentage of students that must pass the tests rises over time precisely to ensure that schools don't focus just on the kids most likely to pass. Also, gotta ask...if all teachers were dutifully attending to all students in the pre-NCLB/pre-standards era, why are we in the jam we're in? NCLB's far from perfect but its critics do themselves no favors when they romance the pre-NCLB status quo as if it were utopian with rich curriculum for all kids, attention to all, etc....
I will go out on a limb and predict that this will be a disaster and possibly of epic proportions. The AFT is struggling to find its bloggy voice (but getting more interesting to read) and the NEA is not the AFT either culturally or in terms of visibility...
It's beyond me how the AFTies will ever "fix" No Child Left Behind if this is how they spend their time. Nonetheless, some interesting numbers, a fun website, and a good spellcheck catch in their latest post about the blog you're reading now...but in their fetishizing they ignore the key stat: Percent of AFTie NCLBlog posts mentioning or about Eduwonk? Somewhere between 20 and 25 percent! Hmmm...maybe less time on this blog and more on signature gathering? Just a friendly suggestion!
Anyway, while you're over there, One-L does point out that the AFTies are not on the big organizational sign-on letter to overhaul NCLB and haven't joined the NEA in suing over the law. This is actually interesting because it's a toe dip into the NEA v. AFT water and the AFTies tend to be reluctant to point that stuff out in public...
It doesn't stop. Now over at Edspresso Matt is turning to sarcasm! If his point is that spec ed is too adversarial for un-empowered parents now, then I don't see how introducing a component, vouchers, that seems sure-fire to increase the adversarial relationship between school districts and parents over special education placements helps. If school districts are identifying too many kids (which Matt and I think they are) for special ed then tying vouchers to special ed placements could well introduce a disincentive for school districts but create a corresponding incentive for parents. And, while school districts are identifying too many students as having learning disabilities that obviously doesn't mean that every student identified for special education with a learning disability is wrongly identified. So a blanket disincentive doesn't make a great deal of sense either.
But Matt says so far there is no evidence from Florida that this is happening with McKay (look for an ES analysis of the program later this year) which then begs the question about how this creates a disincentive and isn't just a run of the mill voucher program? In other words, if McKay were doing what its proponents say we'd see (a) kids with more severe disabilities disproportionately served and (b) a reduction in identification So far neither of those things are happening. We're starting to repeat ourselves here so let me conclude by saying that if the goal is more choice then let's have that debate, but special education has enough problems without using is as a subterfuge for school vouchers.
The SCOTUS announced this AM (pdf) that it will hear two cases (Seattle and KY) on race and K-12 assignment. Significant. SCOTUSblog here and AP here. Update: NYT's invaluable Greenhouse here.
WaPo's Keating and Haynes turn in one of the paper's periodic exposés on the scandal that passes for special education in Washington, D.C. More data on the kinds of students served, rather than just their cursory demographic overview buried toward the end of the story would be helpful but the article does a great job making clear the contours and severity of the problem (handy charts here). In terms of the ongoing McKay back and forth, two thoughts: In defense of McKay, it doesn't necessarily have to lead to a cost-explosion like the one in D.C. Keating and Haynes show why the District's program is so out of control on that front. But, it does speak to the perverse-incentive issue. Because the District's system is so easy to game it does arguably work like a de facto McKay program and a McKay style program in DC would arguably only open the doors more.
What follows is a guest post by Dan Goldhaber on L'affaire Sanders:Having spent the past 5 years or so looking pretty carefully at the impacts of NBPTS certification on teachers and students in North Carolina, I’ve been following the trials and tribulations of NBPTS and the Sanders Three with great interest. Thus, I was happy to oblige when Andy (who clearly needs a nickname – suggestions welcome) generously offered me the opportunity to weigh-in on the debate.
What bemuses me about the entire situation is the way in which the findings from my research work with Emily Anthony [2005] and The Sanders Three [2005] have often been reported (though not on this blog): As if they directly conflict with each other. It’s endemic in the press and policy circles to boil research down to its simplest form — thus, when findings are shorn of their original nuances and caveats (remember, we are talking about researchers, so there are lots of these) and condensed into sound bites, they can frequently appear to be contradictory when in fact, that may not be the case.
So what are the differences between our respective studies?Let’s begin with the data. There are several differences in the study samples: the Sanders Three focus on two large districts from 1999-00 to 2002-03, while we use a sample that includes all districts in the entire state from 1997-98 to 1999-00. The Sanders Three examine grades 4-8, whereas we focus on grades 3-5.There are also some differences in our methodologies and variables used (we included quite an extensive set of student, teacher, and school variables that we believed to be correlated with student achievement) as well as the comparisons we make. It is on this point that I take issue with the Sanders Three paper, as it claims that their hedonic linear modeling (HLM) methodology is superior. HLM is a terrific methodology to use when accounting for the nested nature of educational data (in other words, it can account for the fact that students are clustered in classrooms, so an event that effects one might effect all of them), but it’s not the only methodology that does this, and it does have some shortcomings. In particular it does not allow for the possibility that the error term is correlated with included variables. This is quite likely given that teachers are not randomly matched to students. For example, parents who help and encourage their children at home may also get them into “better schools” or assigned to “the best” teachers. A significant amount of research does in fact show that higher achieving students are more likely to be taught by more credentialed and experienced teachers. That said, correcting for this possibility (which Emily and I do with school and student fixed-effects models) tends to decrease the magnitude of the NBPTS effect (implying that it is unlikely that the Sanders Three non-NBPTS findings result from a failure to address this potential source of bias).
So what’s the bottom line here? There may well be differences between the two sets of findings, but it’s hard to tell given that we focus on different samples and employ somewhat different methodologies. My guess is that the differences are more a question of emphasis than of substance. Emily and I find more “statistically significant” NBPTS effects, but it’s important to remember that a statistically significant finding doesn’t mean that an indicator is going to be a perfect predictor of an outcome or that statistical significance equates with substantive significance in policy. College graduation is certainly a predictor of future earnings, but there are a lot people who didn’t graduate from college with high salaries, and there are also plenty of unemployed college graduates. Similarly, a statistically significant NBPTS finding does not mean that all (or even a large majority of) National Board Certified Teachers will necessarily be more effective than non-NBCTs.
That said, I believe that the Sanders Three emphasis on the within-category (NBPTS vs. non-NBPTS) variation in teacher quality and the overlap in the distribution of teacher effectiveness is absolutely correct. I conducted some follow-up research (which will be published this summer in Education Finance and Policy) that also found a significant amount of overlap. Specifically, I found that NBPTS teachers would be predicted to outperform non-NBPTS teachers around 55-60% of the time—better than a flip of the coin, but certainly no slam dunk when it comes to choosing the better teacher. This issue of overlapping distributions is coming up more and more frequently (see, for instance, Gordon, Kane and Steiger, 2006, or Goldhaber, 2006) and it ought to inform the way we view teacher policies.
So, is it worth investing in the NBPTS model? I think it's still too soon to say but I’ll leave that to the policymakers. But I do want to suggest that we try to move the policy discussion beyond the well-worn ‘NBCTs are better/worse than non-NBCTs’ debate. We ought to be concerned about the effects on different types of students, whether NBCTs have effects on students other than those in their classrooms, and whether there are impacts apart from those we can detect based on student test scores. There is a great deal of research on NBPTS that should be coming online soon, which will hopefully shed more light on the value of this credential in different teaching contexts. It might also provide information that the National Board can use to improve their process. Coming back to my original point, we would probably be better off if the policy debate moved from it’s “good” or “bad” to “how can it be made better?” --Dan Goldhaber