Wonks Gone Wacko

15 Replies to “Wonks Gone Wacko”

1. I realize this particular blog entry may be a total bust, but I thought I’d give it a whirl.

   There are three parts to this production function.

   Part One
   Classroom Learning = Time * “Misbehavior Tax” * Individual Kid Effort *
   Quality of Class

   Part Two
   Out of Classroom Learning = Time * “Legit Desire To Learn” * Quality of
   Assignment * “Distraction Tax”

   Part Three
   Kid’s School Learning = CL + OCL

   EXAMPLE OF CL

   How much will a kid learn in an 8th grade math class, given the inputs above?

   He probably has an hour-long math class each day. So 180 hours per year. Minus assemblies and other silly stuff, maybe 165 hours per year.

   However, the class is taught by a second year teacher who is still
   pretty lax with discipline. An observer who counted the minutes lost to
   kids screwing around, talking, etc, would say “only 30 minutes were kids
   either observing/listening to the teacher or actually doing work at
   their seats.”

   So the misbehavior tax is 50%. That’s the second variable. In other words, even a kid who is inclined to try hard only gets 30 minutes of “real math class.”

   Now imagine three different kids.

   Kid A works really hard – 100% effort, even if she doesn’t always
   understand what is going on.

   Kid B is mostly on task, but 20% of the time she is daydreaming, or
   trying to frantically study for science quiz next period, or whatever. Then she tunes back in.

   Kid C has his head on the desk, napping, much of the period. He is also
   “lost” – just confused because the math builds on concepts he doesn’t
   understand from previous lessons. In a typical class, he is
   legitimately trying for just 10% of the time.

   “Quality of class” is the fourth variable. That captures a ton of
   elements. What is the relative import of the topic? Does the teacher
   have kid mindlessly crank out problems that they don’t really understand
   simply by mimicking steps, or does she elicit some level of legit
   comprehension? We can go on and on.

   Let’s just give that an arbitrary value on a 1-10 scale. I bet if you
   and I observed the class we’d rate “quality” fairly closely. I’ve
   actually done this with tons of people, never defining terms, just
   saying “what is the quality of this class on a 1 to 10 scale?” and we go watch a class and then rate it.

   Generally we end up in the same ballpark. I might say 7 and you say 6. But rarely you say 2 and I say 9. Let’s just leave “quality” definition alone for a little while and take our “quality rating” at face value, that we know it when we see it, although obviously we could debate that point forever.

   So

   Classroom Learning for Kid A = 165 * 50% * 100% * 70% = 57.7 arbitrary CL units.

   Classroom Learning for Kid B = 165 * 50% * 80% * 70% = 46.2.

   Again, 46.2 what? CL units. It’s not important. Right now, we just want to be able to COMPARE how the inputs affect the generation of CL units (output). If we can increase CL, the kid learns more math.

   I’d try to steer your attention to “Misbehavior Tax.” That is a very
   powerful variable.

   Holding all other things constant, if you can just get the distraction down from losing 50% of the minutes of a class, to 25% of the minutes of a class, look at the effect. Kid B’s learning for the year goes from 46 units to 69 units.

   In other words, in a classroom where the Misbehavior Tax is held down,
   Kid B (the 80% kid) will actually learn more than Kid A (the 100% effort
   kid) in the original classroom.

   Veteran observers of schools will note that same exact 20 kids will go from one class where they behave well to another class where they will screw around. There is specific teacher effort and skill involved. Usually it involves 1-on-1 relationships with each kid and a steady, consistent, even-toned approach to discipline. But anyway. When unchecked, this tax is very powerful!

   I think Ed School types focus on “Quality of Class” because it’s the most intellectually interesting.

   Policy types often talk about “time” — extended day, extended year — because it’s an easily controlled variable.

   I believe the most upside — the key ways to get kids to learn more — is in these two less explored variables, particularly lowering the “Misbehavior Tax” on learning.

   Thoughts on the production function? Anyone? Bueller?

2. GGW,

   The last two posts were what I was looking for. I’m absolutely sincere in my admiration of the South Bronx Charter School and your Wonks Gone Wild. We say it differntly, but at Core Knowledge, in Union formats, etc. we are talking about the misbehavior tax. Maybe I’m wrong, but often (usually) those uplifting pieces seem to imply that we could get the same results at Locke, Douglass High in Baltimore, or my school, when that is patently absurd (and I think that most people who make that political arguemnt also know that it is a political argument that they make in order to help poor schools). But we are in a sum zero game and the freedom that choice schools have is the result of a trade-off and the practical effect is to veto efforts by equally effective and committed educators to turn around our neighborhood schools. I loved the video’s discussion of Black and White vs. Gray. But in neighborhood schools we can talk but we don’t have any choice but to navigate our shades of gray. If you do not mean the video as a political attack on us, then I apologize for suspecting that.

   There are two phrases that I would like us to stop using for the sake of politeness, if nothing else. The first is the phrse “Whatever It Takes.” In a neighborhood school, it teaches young teachers the wrong lesson. We have to be like emergency room physicians who do their best but if we want to serve kids for the long run, we need to learn to mourn.

   The second is: “Veteran observers of schools will note that same exact 20 kids will go from one class where they behave well to another class where they will screw around. There is specific teacher effort and skill involved. Usually it involves 1-on-1 relationships with each kid and a steady, consistent, even-toned approach to discipline.”

   Of course that statement is half true, but it causes more harm than good. There is no comparison to my job as a veteran. when teaching upperclassmen, in our Black History, Multiculturalism, and Government classes to a young teach with no administrative backing while teaching Freshmen Algebra or French. Too many great teachers are like Ted Williams who was a superstar but who wouldn’t widen his strike zone or sacrifice himself for the team.

   We this is now as long as my original thoughts, but they are related, and yours was pretty long also.

   So,

   I just finished Relentless Pursuit. Donna Foote does a fantastic job of portraying an urban neighborhood secondary school after the easier-to-teach students have been creamed away, and a long series of crackpot reforms has created an immunity to change.

   We won’t have progress until we can efficiently get rid of the worst teachers. But that will accomplish little if we can only replace them with longterm subs. To recruit and retain more teaching talent we must find a way to address the disorder that is created by kids who are in pain. I don’t know what the solutions are but they must be humane and in the best interest of those kids, but we must have the gumption to efficiently remove them to other sites when necessary. That means high-quality alternative schools but that is not enough. I wish I had a full answer. But I do know I would love to have a principal like Frank Welles.

   I loved most of the TFA’s approaches. More important than improving teacher training is recruiting a higher quality of teaching talent. Teachers must primarily be leaders. Similarity, after receiving reality check, TFAers in the book realized the need to build relationships. A relentless focus on instruction is necessary, and in some cases it can be sufficient. But systemically, we must build relationships, create safety and order, and address social conditions before the curriculum driven approach can turnaround the toughest schools.

   Some TFAers have a cult-like faith in Objective Measurement, but it was reassuring to hear how many did not buy into the extreme numbers-driven approach, and that TFA is constantly upgrading. It is one thing to create a data-driven system to recruit a few thousand teachers from a narrow spectrum of society, but it is another to believe that test-driven accountability can spearhead national reform. If your purpose is to garner business support, advance your career, or attack political enemies, then their fixation on numbers is rational. But if your purpose is to help children – and I’m convinced that is the real purpose of TFA – then some of their leaders have to get over their zealous faith in data-driven accountability and respond to the human factor.

   If you want to help kids, you must work with people who do not share your religion of test-driven accountability. If they are going “to play well with others,” they must stop challenging the sincerity of educators who support data-driven decision-making but disagree with high-stakes testing. For SOME TFA leaders described in the book, their scorn for their opponents was like a person who wouldn’t vote for Obama because his name is Barack. They might as well be wearing the tee shirts that were popular in the first naive days of econometrics, “if you don’t count, you don’t count.”

   I wish the TFA would amend their principles to include “Collective Responsibility.” But they need to define the word Collective in a Broader and Bolder way.

   We need to remember the old maxim, “Its better to have everyone in the tent peeing out, than have people outside peeing in.” If none of that yellow stuff dumped on us comes from you, please encourage your friends to consider my suggestions.

3. In your current formula, variables have no separate weighting, and you have limited numbers of variables. Where are additional variables such as “size of vocabulary when entering school”? Quality of class alone could be broken down into multiple variables. And some of the variables are not independent — for example, quality of class and misbehavior tax are likely strongly linked.

   My gut reaction: too simplistic to be of much value.

   Parry

4. I agree with Parry — the quality of the class and the ‘misbehavior tax,’ or classroom management/lack thereof, are inextricably linked. You have a boring, teacher-centered, listen-to-me-talk class, you will get misbehavior from kids who do not have it culturally beaten into them that they must sit quietly and listen to the tall person at the front of the room. On the other hand, you have an intellectually stimulating, explicitly relevant, student-centered class, you will still get misbehavior — but significantly less.

5. Yes, it’s simplistic but I’m assuming that Michael is doing this to show teachers what their contribution is to student learning. Most of them don’t or can’t see it and so they divorce themselves from the outcomes of their effort.

   That said, you’re weighting too much on the student and not enough on the teacher.

   The research from ACT and Ed Trust on diverse, high-performing high schools (On Course for Success) calls attention to teacher pacing. It’s kind of obvious, but effective HS teaching doesn’t repeat content learned in previous grades, doesn’t waste a lot of time taking attendance and going over last night’s homework, all which help teachers get good coverage of the standards.

   The misbehavior tax and student effort are highly collinear. That is, the kids who misbehave are also highly likely to not put in much effort. You probably could drop one.

   Finally, why is the classroom the only unit of analysis? What’s the school’s contribution to performance?

6. A very nice effort! Your formula places a key variable at the center of the matter: time in classroom spent learning.

   In my visits to grossly underperfoming schools, one thing that really popped my eyes was how much time teachers spent trying to keep order in the classroom. Often, the better part of a 45 minute period was spent telling Johnny to “please sit in your seat,” and Susie “to quit talking.” Is it any wonder that achievement levels at these schools were so low? A school where unruliness is the norm seems unlikely to be a school where most kids will learn much.

   The big question is: how to train teachers so that they are equipped to handle unruly students, to stop a slightly boisterous classroom from tipping into anarchy?

   Cheers,

   KRK

7. Hi Parry,

   Thanks for your comment. So, keep in mind, I’m narrowly measuring “Classroom Learning.” Another way to say that is NOT “Total Academic Skill Level” but only “GAIN in Academic Skill Level over baseline.”

   So another formula which I did not show, but perhaps should have:

   Students Academic Level = Baseline + Gain Over Baseline.

   Vocabulary level entering school is HUGELY powerful. But assuming a bunch of kids who have very low baseline vocabularies, the job of a school is to generate the Gains. Certainly, meanwhile, “society” can work on the Baseline.

8. Classroom Learning = Time * “Misbehavior Tax” * Individual Kid Effort * Quality of Class * Student Aptitude

   Several observations:

   1. Time and effort are going to have some sort of relationship to each other, in that past a certain amount of time, there might be some burn out and reduced effort.

   2. You completely leave out student aptitude (some kids take less effort to get to the same point)

   3. The misbehavior tax is partially related to quality of teaching. Kids often misbehave out of frustration from poor teaching. The misbehavior tax could be significantly reduced if reform starts at the earliest grades. Zig Engelmann talks a lot about this misbehavior tax in his recent manuscript on the history of Project Follow Through and Direct Instruction.

   4. If the misbehavior tax is that critical, doesn’t it lead to the conclusion that misbehaviors should just be excluded from classes with non misbehaviors?

9. David, you said what I meant better than I did.

   Veronika, actually, I think those can be separate variables, Misbehavior Tax and Quality Of Classroom.

   In other words, imagine 2 teachers with the same lesson plan.

   To use your phrasing, “boring, teacher centered.”

   Would you agree it’s possible to imagine very different levels of misbehavior among those kids, holding the lesson plan constant, with simply different teachers?

   Or even imagine your preferred “student-centered” lesson plan. This time, pretend it is taught by the same teacher to similar kids in 2 different schools, with very different schoolwide cultures.

   Again, can’t you imagine very different outcomes in terms of misbehavior?

10. Rory,

    1. Yes. So let’s say you created a 4-hour per day math class. Time would rise. But kid effort would plummet. Misbehavior Tax would increase. Some elements of “Quality of Class” would decrease.

    So if we search for optimal time, we have to balance all the factors — all other things being equal, more time = more learning, but all other things are not equal, which I think the formula accounts for.

    2. Per above, perhaps I should have had said “Let’s try to create a production function that measures learning but only examines factors that the school can at least partially influence. Two factors schools cannot easily influence are hereditary aptitude and baseline academic skills.”

    3. Thanks for recommending the Zig book. I will check it out.

11. John,

    Thanks for your thoughtful comment. Lots of interesting observations you make.

    One small note: I was careful to note “Same 20 kids” — ie, 20 kids who walk from 9th grade math down the hall to 9th grade science. The same 20 kids wouldn’t walk from Freshman Math to your upperclassmen class on Black History, right?

    My point there was that holding all other factors constant (lesson plan, kids), some teachers — and some schools — are more effective at lowering the Misbehavior Tax.

12. You’re right, but my point holds also. I can take the same freshmen kids and feel smug about my success because I can teach concepts that are more interesting and relevant to them than the Algebra or French class. The fact that I’ve been teaching the kids’ older brothers and sisters for years, and playing basketball with them, and creating a 16 year old tradition so that students know when they walk in the door that OUR class is different, doesn’t give me an advantage? I might convince myself I’m a superstar, but politics pervades the situation. Being a veteran teacher, I can and will say screw the rules that come down from the central office on curriculum pacing. During a presidential election, we discuss presidential politics, and during the Supreme Court session we discuss the Bill of Rights, with special focus on the issues that directly effect the students. The new teacher wouldn’t dare put their professional judgement over top down mandates, no matter how stupid the policies are. Politics will determine that the Algebra teacher will get a favorable class load, but the toughest kids will be dumped on the French teacher. After all, we never get a French teacher who comes back for a second year. Having proven my classroom management skills I can get disciplinary backing when I demand it, but the young teachers and the less charismatic teachers are on their own. The same applies to principals, of course. It is politics that determines which principals are allowed to enforce their policies. Often, perhaps usually, merit is involved. Effective teachers and principals get more backing. But that just dumps even bigger challenges on the struggling educators. And often, it is just politics.

    The biggest factor is critical mass; it is the percentage of traumatized kids in one location.

    As I will explain in the Quick and Ed, our school and another of our three neighborhood high schools just tied for the lowest ACT scores (15.3) in urban Oklahoma. But hundreds of poor Black kids in our zip code attend a school that is ranked in the top 50 nationwide, and several attend a school that is typicall ranked in the top five (31+ average ACT). There is no comparison between our weakest faculty and the weakest of those high flying schools. Year in and year out, about of a third of our classes have subs or people who should have never been allowed in a classroom. But compare the top third of our teachers, and perhaps the top 2/3rds and we are every bit as good. After all, a large number of their teachers left us in exhaustion, and came back to life when they were allowed to teach without the obstacles we face.

    Our young talent is just as good as theirs, but it is virtually impossible for a young teacher to survive in our school. (even though we have two that are awesome) Why? the complete lack of disciplinary backing is the prime reason.

    I admire teachers, young and old, neighborhood or magnet, who find a way to be effective. I understand, given the stressful nature of our job and the ego investment involved, why teachers who are successful will distance themselves from teachers who are less successful. But on the other hand, I always wonder why – if they are so confident – they are so quick to judge others. Of course, I might just be talking noise also.

    Its easy to claim that any of us are working miracles. We need data driven decision-making as a corrective. But if people like the Ed Trust were so confident that their policies can raise all boats, why do they use so much bogus data? Why not apologize about their most discredited studies?

    And if we really want to help poor kids, why get stuck in this cycle of recrimination? How are we going to recruit new talent to a system that is so good at eating its young? Lets put more work into the hard work of improving schools and less in distributing blame.

13. Umm, you realize that this time formula was set out in 1963 by John Carroll, Carroll, J. B. (1963). A model of school learning. Teachers College Record, 64(8), 723-733.

    However, this work has been built on rather extensively by other researchers who have shown, rather convincingly, that time is not just time. The Carroll model, like the Goldstein model here, is too simplistic for application in schools.

    Cooley and Leinhardt, and David Berliner, wrote about the differences in how time was used in instruction. Basically, all time spent in a math lesson is not equal. So is better to have an hour long math lesson taught poorly, or a 30 minute lesson taught well?

    There is some evidence that time matters a lot in terms of school year, and there is evidence that for disadvantaged students more time in school each day may be less about teaching and learning, and more about avoiding some other activities that are detrimental to learning.

14. Late to the party, as usual, but I came here to say what “Cooler Heads” said very nicely. I would just add that you probably need interaction terms. . .that is, the cost of a given level of distraction varies, depending on the value of other variables.

15. Hi Cooler,
    Welcome to the party. I think I differ a bit from Carroll. He posits 5 factors.
    a. Two we agree on. Time. Quality of Instruction.
    b. Two of his (aptitude and “ability to understand”) are, from the point of view of schools, tough for us to affect.
    c. Perseverance is, I think, similar to student effort, but his article seems to think of perseverance as fixed, whereas our teachers find student effort can be significantly affected by the teacher.
    d. Nothing about the “Misbehavior Tax” – which I would argue is the dominant feature in high-poverty middle and high schools.