"Least influential of education's most influential information sources."
-- Education Week Research Center
"full of very lively short items and is always on top of the news...He gets extra points for skewering my high school rating system"
-- Jay Mathews, The Washington Post
"a daily dose of information from the education policy world, blended with a shot of attitude and a dash of humor"
-- Education Week
"unexpectedly entertaining"..."tackle[s] a potentially mindfogging subject with cutting clarity... they're reading those mushy, brain-numbing education stories so you don't have to!"
-- Mickey Kaus
"a very smart blog... this is the site to read"
-- Ryan Lizza
"everyone who's anyone reads Eduwonk"
-- Richard Colvin
"designed to cut through the fog and direct specialists and non-specialists alike to the center of the liveliest and most politically relevant debates on the future of our schools"
-- The New Dem Daily
"peppered with smart and witty comments on the education news of the day"
-- Education Gadfly
"don't hate Eduwonk cuz it's so good"
-- Alexander Russo, This Week In Education
"the morning's first stop for education bomb-throwers everywhere"
-- Mike Antonucci, Intercepts
"…the big dog on the ed policy blog-ck…"
-- Michele McLaughlin
"I check Eduwonk several times a day, especially since I cut back on caffeine"
-- Joe Williams
"...one of the few bloggers who isn't completely nuts"
-- Mike Petrilli, Thomas B. Fordham Foundation
"I have just three 'go to' websites: The Texas Legislature, Texas Longhorn sports, and Eduwonk"
-- Sandy Kress
"penetrating analysis in a lively style on a wide range of issues"
-- Walt Gardner
-- Education Week's Alyson Klein
-- Susan Ohanian
Smart List: 60 People Shaping the Future of K-12 Education
29 Replies to “Another One For The KIPPer”
Yet another less than stellar report on the benefits of charter schools. The crowd goes wild!
“Less than stellar report on the benefits of charter schools,” Mr. Calder?
When you manage schools that draw the following judgment from research such as Mathematica’s, come and post it, and we’ll go wild over you!
Students coming into KIPP schools had prior achievement levels that were lower than average in school districts.
“For the vast majority of KIPP schools studied, impacts on students’ state assessment scores in mathematics and reading are positive, statistically significant, and educationally substantial.”
Impacts frequently are “large enough to substantially reduce and race and income achievement gaps within three years.”
would the learning be to pay the group that does the research if you want good news?
unfortunately as long as the KIPP Foundation is paying Mathematica to do the research, they’ll probably get the results they want.
if the aft or the nea paid for the research, what would it highlight?
maybe the lack of special ed or english language learners in KIPP schools? who knows?
Sandy don’t stress your self. Steve et al., have imposed an impossible burden of proof on these studies. The fact is no one could design or conduct a study that would satisfy them.
Here’s an exercise: Someone who doesn’t’ like this study, please describe one that would satisfy you.
Before I make my point I want to say that from what I’ve read, the KIPP schools seem to do an excellent job. Perhaps I would have placed my own children in such schools. I admire and appreciate any school that offers at-risk children a better chance at a quality education. I am grateful to the foundations that donate their money to such schools. And yes, I do agree that there is something to learn from the success of these schools.
However, when we talk about “research” that is another category altogether, especially in regard to the social sciences. There are just so many variables, that it’s virtually impossible to get the kind of results that are valid. One thing I learned when I was in graduate school is that almost every “study” in education came out the way the researcher wanted it to come out. This doesn’t mean they cheated; it just means that there are so many variables that it’s easy to guide the study along a certain desired path. So I’d like to describe two studies that might satisfy me:
I’d like to see an independent, outside researcher (NOT one hired by KIPP) study a KIPP school that took over the entire school population of a low-performing school and compare it to a KIPP school that started from scratch and required parents to apply. Ideally these schools would serve children in the same, or nearly the same, community. Were there significant differences in the levels of achievement? What were these differences? What are the possible causes for them?
Another good study might focus on two traditional public schools in a low-income neighborhood. Both schools have similar demographics, yet one can demonstrate a high degree of learning, while the other can not. I’d like an outside researcher to come in, study the two schools and try to isolate the significant differences between them.
Testing for both studies would be conducted by an independent testing company that would administer, collect and score the tests. These tests would reflect the curriculum taught but would not be seen by school personnel before administration, nor handled by them at any time.
Those of us who are somewhat skeptical about the Mathematica research might not be experts, but we know apples and oranges when we see them.
Linda are you familiar with Merseth’s book? http://www.hepg.org/hep/book/94
No, I am not familiar with the book. Thank you for the reference.
“However, when we talk about “research” that is another category altogether, especially in regard to the social sciences. There are just so many variables, that it’s virtually impossible to get the kind of results that are valid.”
That’s why they invented MULTIVARIATE (hence multiple variables) regression. It has the statistical power to control for co-variates (all those other variables you reference) and isolate the power of a key explanatory variable relative to other influences on achievement. That said, observational studies are never as good as experiments. And that get’s to your other point below:
“I’d like to see an independent, outside researcher (NOT one hired by KIPP) study a KIPP school that took over the entire school population of a low-performing school and compare it to a KIPP school that started from scratch and required parents to apply. Ideally these schools would serve children in the same, or nearly the same, community. Were there significant differences in the levels of achievement? What were these differences? What are the possible causes for them?”
That’s what researchers do when they do a randomized field trial studying achievement in a lottery of students. The only difference is one set of students were “treated” with being plucked out to go to the KIPP school and the other students were not. Both groups of students are similar on all other key demographic factors – including parental contracting and involvement – obviously non lottery winners were just as engaged as lottery winners in putting their kids in the lottery. These randomized trials are good enough for studying medicine at the Mayo clinic, so they should be good enough for studying social science on learning gains.
“The only difference is one set of students were ‘treated’ with being plucked out to go to the KIPP school and the other students were not.”
This is not the only difference to me. The chosen KIPP students are grouped with other students whose parents also selected KIPP while the students not chosen presumably went to a variety of traditional public schools. So the critical variable could have been being in a class with all students having involved parents.
Merseth’s book was the least critical trash I’ve ever read. A huge disappointment. Linda, you wouldn’t send your kids to KIPP – you don’t want them to be schooled into nonthinking rulefollowers — it’s only good enough for certain kids, but the corporate bigwigs wouldn’t be pleased if their kids were institutionalized in such a manner.
“This is not the only difference to me. The chosen KIPP students are grouped with other students whose parents also selected KIPP while the students not chosen presumably went to a variety of traditional public schools. So the critical variable could have been being in a class with all students having involved parents.”
1. Do you know of any studies that link improved student achievement (not baseline test scores) but academic growth at the individual student level to the contextual effect of going to school with peers who have more involved parents? I know more involved parents have been linked to improvement for the children of those parents, but was not aware of a scholarly literature demonstrating contextual effects for peers’ parental engagement.
2. Even assuming 1, your point just demonstrates that KIPP has an effect – though the influence of KIPP is black boxed – you would claim its because it creates an extraordinary environment where the norms of parental involvement are inculcated in such a way that spills over to all students in that schoolhouse (even the ones whose parents are least involved. The point still holds that students who were not fortunate enough to win the lottery did worse, even though they entered the lottery at a similar place in life (similar test scores, similar levels of parental engagement, similar desires to succeed).
At the end of the day our disagreement is still within the black box known as KIPP. You would attribute (most? all?) of the academic growth of lottery winners to the benefit of being surrounded by kids whose parents are engaged. Whereas, I would say that given the literature on what school-level factors influence growth of student achievement its likely that the teaching quality at KIPP schools (or availed to Lottery winners) is the main causal factor at play.
if you lived in a public school district and your child had a choice between KIPP (with 80 percent plus college graduation rates for its students) and a neighborhood assigned public school where fewer than half of the students graduated, for your kid’s sake i would hope you wouldn’t let ideology get in the way of their future.
after all, even members of congress and the unions who don’t like school choice and charters still seem to send a lot of their kids to private schools. 🙂
Statsgeek, I’m not a statsgeek, so this discussion is now above my pay grade, as the president might say. I was a first-grade teacher for many years, so I’ll give a simple answer:
If a tobacco company hires a research company to evaluate the safety of its product, I am going to view the results of that study with a grain of salt. However, that does not mean that the research is wrong; it just means that I am going to keep an open mind. And that’s how I feel about the KIPP research.
Josh Angrist and colleagues analyzed achievement in a KIPP school in MA and found that kids in the KIPP school outperformed a comparison group of similar students. The comparison group was students who applied for the lottery but were not lotteried in, so it’s as much an apples to apples comparison as one can make outside of a randomized experiment. This KIPP school is particularly interesting because it serves a large number of Hispanic kids and also kids with special needs and limited English proficiency.
The research team reports that “LEP students, special education students, and those with low baseline scores benefit more from time spent at KIPP than do other students, with reading gains coming almost entirely from the LEP group.”
@statsgeek & @Linda/RetiredTeacher:
“You would attribute (most? all?) of the academic growth of lottery winners to the benefit of being surrounded by kids whose parents are engaged.”
I’m not sure if any studies have been done on the peer-effect factor, but anyone who has been in a classroom knows that it does exist. A student can be a living terror and non-achiever in one classroom, but when moved to another with a different group of students who don’t appreciate the antics settle down and do very well (we’re assuming teacher stays the same, only peer group changes).
“given the literature on what school-level factors influence growth of student achievement its likely that the teaching quality at KIPP schools (or availed to Lottery winners) is the main causal factor at play”
You have eliminated the effects of the extra parental involvement required, the much stricter discipline at KIPP schools, and the increased instructional time? (I’m noting that most of these would be problematic at a school taken over by KIPP.)
look we can go back and forth about why KIPP schools do what they do. But in so far as one can ‘prove’ anything we know that, students admitted to a KIPP school perform better than they would have if they were not admitted.
Given that, the question becomes, should we:
a) continue to limit the expansion of these schools
b) try to expand these schools so they can admit more students.
This is and this isn’t about KIPP.
KIPP can be as great as it wants to be and it will. But that will not drastically change the education landscape in this country – the model is limited! KIPP could not possibly run more than 500 schools and that’s a generous bet on my part. There are limiting factors that will always hamper its growth.
I’d love KIPP to open more schools – they won’t though, they care to much about their image to expand too quickly!
The overworked young talent will not last forever, the cheap space and reform friendly atmosphere will not last forever. KIPP is not about education – it is about achievement, so when achievement swings to a new metric so will KIPP’s cherished results.
With all the factors working for KIPP, it might be worth asking why it isn’t more successful? The kids have drastically longer schools days, filled with like-minded peers, and “effective” teachers – shouldn’t the scores be out of the ballpark?
A real education cannot be easily measured, it takes years for one to be considered well-educated, indeed it may take a lifetime.
I think we’re all reaching a kind of agreement that education is a very complex endeavor. Yes, the KIPP schools DO appear to get good results, but the “why” is not so obvious to many of us.
This morning I was thinking about how our beliefs are so much better understood by our actions, and not our words (not an original idea, I know). If you want to know what you truly believe about education, ask yourself this question: If you had a choice, would you send your child to:
a KIPP school
the closest public school, even if the scores are low
the best private school you can afford
a school with the highest number of high-achieving students, regardless of whether it’s public or private.
If you answered one of the last two choices, you know, as many people do, how critical the student population is to your child’s success. Simply put, if he is surrounded by high-performing, well-behaved kids who value education, he will probably become high-performing also. The teacher will teach to the level of the class. An average teacher will almost always achieve good results with high-performing students.
KIPP schools, in my opinion (might not be so) take a step in the direction of limiting the type of students in the school. This is not a bad thing but it should be recognized for what it is. Once we face the truth of what makes a school successful, we can look at those factors and then do what we can to make them available to all children.
For example, once we recognize the importance of the student factor, we’ll find ways of moving children out of high-poverty areas (low income housing in all communities, jobs, public school vouchers, private school scholarships, boarding schools, etc.).
If we truly want to help other people’s children, we need to keep an open mind to all the possibilities. When I was teaching, if I ever became uncertain about a decision I had to make, I always asked myself, “What would you want for your own child?” The answer was always there.
Steve your response is laughably hypocritical.
You: There’s no such thing as a silver bullet in education
You: KIPP is not a silver bullet, therefore, KIPP is a failure
Sorry for the long reply, Bart.
I’ll keep my position brief.
Me: Let’s not waste too much time discussing KIPP. It is an institution of the current education policy culture, which if history teaches us anything will change.
Instead, let us restore our public schools for all kids not lottery applicants or lottery winners.
steve, there wouldn’t be lotteries if the unions would stop blocking legislation to take the caps of charter schools.
I’m sorry, I’m just too pessimistic.
I don’t see a world in which charter schools would take all kids – why would they want to be “accountable” for that?
why think in theoretical terms? which type of schools are more accountable today, charters or traditional public schools? charters can have their charter revoked, generally receive less funding per pupil compared to traditional public schools, and still have to meet all state/federal accountability requirements. outside of a handful of states that are aggressive with takeovers, most traditional public schools can’t lose their “charter” or be shut down for poor performance. do charters have problems? sure. but on the accountability measure as it stands today its an easy call charters > district schools.
you’re thinking the same thing i am, but reaching a different conclusion. charters can be held to a higher degree of “accountability” so they operate in different ways and don’t take all students.
as i said i don’t imagine a future where charters will ever take all students. for the simple point you make – it’s so easy to hold them accountable and put them out of business. why would they take that risk?
what would an urban district with 75% charter schools and 25% traditional district schools look like? what kinds of kids would be in those two types of schools. i’d be interested to hear your thoughts.
if i am not mistaken, by law, charter schools have to accept all students. now, in practice, i will readily admit that schools like KIPP require students to sign an agreement to abide by a contract. the terms for expulsion certainly differ from a traditional public school. that said, i haven’t read any reliable accounts of mass expulsions from kipp in which half of the entering class gets the boot for being 5 minutes late to class.
at the end of the day no school will be able to help students whose parents are crackheads and won’t open the mail to learn about a charter lottery or even that the law requires them to get their kid to school period. until we as a society say that kids can be taken from parents from neglecting their education that eggregiously, i’m satisfied to allow a charter market to bloom that can absorb and be accountable for the vast majority of students whose parents do want to get them a good education but can’t because there are no high quality options in their assigned schools.
i admit in some ways my answer says the poor will always be with us… (your 25 percent) but i think it would be far less than that. i dont think kipp succeeds because it kicks kids out – i think it succeeds because it uses longer days, more flexibility in regard to its human capital, and higher expectations. not self selection. that’s what the randomized studies are for… it should be replicated. it isn’t a silver bullet, but it is hard to generalize to a hypothetical world about things like charter growth’s impact on public schools and competition. we still have small market share in the charter industry due to caps.
interesting dialogue though.
thanks for your thoughtful comments, thehitch.
i understand the appeal of charter schools and giving kids in terrible situations a high quality option. philosophically, i struggle with the fact that we have to get the private or non-profit sector involved, deregulate education, and socialize the risk. it usually doesn’t end well for the lowest of our society and those are where my concerns will be focused.
what does socializing the risk mean in this context?
Here’s what I want for my children:
• be lifelong learners
• be passionate
• be ready to take risks
• be able to problem-solve and think critically
• be able to look at things differently
• be able to work independently and with others
• be creative
• care and want to give back to their community
• have integrity and self-respect
• have moral courage
• be able to use the world around them well
• speak well, write well, read well, and work well with numbers
• truly enjoy their life and their work.
(from Dennis Littky)
Now please explain how KIPP’s myopic focus on testing contributes to any of that. KIPP’s formula is simple (I worked in a major charter network school): Test all the time. Multiple choice all the time. High stakes all the time.
If I opened a K-12 school that had basketball all day, nothing else, LeBron wouldn’t know what hit him. I’d have the greatest basketball players in the world and the media would swarm. Public schools realize that we’re developing kids — not machines. The unilateral focus on testing is absolutely destructive. Show me a study of success in college from KIPP kids or grads of similar models. There’s nothing. And there’s a reason why. A kid who only knows how to bubble doesn’t know how to think.