Comments on: I can’t reveal my confidential sources but…

By: Dan Willingham

Dan Willingham — Thu, 21 Aug 2008 15:03:59 +0000

They definitely don’t have the $$ for a large-scale rct. Whether CKF would pay for a lower-cost approach–e.g., data mining–is not up to me, but I think they would very open to a proposal from a competent researcher. They are eager for such work to be done. It is not the ideal situation for anyone to pay for research to evaluate themselves–observers will doubt the objectivity of the research on that basis alone.

It’s odd that someone in the Ed Research community hasn’t taken this on. It’s a decade old and is in 100’s of schools–it should be low-hanging fruit for a young researcher trying to get a grant and make his or her way in the academic world, or for a soft-money type (who must have a constant flow of grant $$) at abt or westat

As to publication, I am sure that CKF would want any results published, but would also want reports to undergo a process of professional peer-review (i.e., via an academic journal).

dan

By: Cooler heads

Cooler heads — Wed, 20 Aug 2008 17:52:45 +0000

Dan, would the Core Knowledge folks pay for such research, and would they allow for the findings to be published simultaneously with their learning of the findings?

By: Dan Willingham

Dan Willingham — Tue, 19 Aug 2008 23:55:16 +0000

Hey
I’m on the board of trustees at the Core Knowledge Foundation and I can verify what noble hart says. . . one of the things I’ve been trying to do is get a researcher interested in such a project (including people at abt and mathematica, as well as academia). A couple of things to keep in mind: for a serious rct with enough schools and classrooms within schools to allow sufficient power and at least 3 years to observe effects, you are looking at a lot of money–probably betwee 750K and a million dollars. The Core Knowledge Foundation doesn’t have that type of money. That means you need to persuade someone who can run such a large-scale study (and there aren’t *that* many of them) to write a grant to IES or a foundation. Persuading someone to devote 6 months to write a grant and then to spend the next three to five years of their career working on a project is pretty hard to do, if the person wasn’t already interested.

noble hart is also right that there have been some studies with positive results. . .but they have been small scale things that have been done with almost no money. . so they don’t meet the standards one would hope for.

The folks are Core Knowledge Foundation would be THRILLED to have any researcher work on an RCT, a data-mining study, or any other scientifically valid method. If you know someone who is interested, send him or her to me!

By: spj

spj — Tue, 19 Aug 2008 21:06:24 +0000

im trying.

By: noble hart

noble hart — Tue, 19 Aug 2008 19:44:38 +0000

Hoxby has independently conducted rct on Core Knowledge with very favorable results. She needs to publish her disaggregated data. The the ed-research world has avoided CK like the plague. Maybe spj will help change that! Hirsch has often pleaded with foundations and IES to run independent evaluations.

By: spj

spj — Tue, 19 Aug 2008 18:14:09 +0000

reading first bombed b/c congress (pushed by the publisher lobby) changed the program from scientifically validated programs to scientifically based programs – huge difference.

i dont buy the idea that core knowledge wants a rct, but cant find anyone to administer it. why dont they give mathematica, abt, spr, or any other big research house and ask them to conduct one? ck isnt short on money. if they are they could ask a foundation to fund it.

evidence and data is all the rage these days, save when it comes to curriculum.

By: john thompson

john thompson — Tue, 19 Aug 2008 17:07:51 +0000

The “gut check” should have made us ask a simple question: would kids benefiet from investments in reading even if the programs were flawed and the documentation remained weak? Or would they benefit more from a flawed test-driven accountability system where the documentation of reliability was weak, and it raised questions of whethert the system could ever be made reliable?

Following the logic that weighing a pig doesn’t make him heavier, wouldn’t we be doing more good to real kids if we were investing more in trying to teach them to read even if the scales for weighing the results aren’t ready?

Common sense says that Core Knowledge would do a far better job than Reading First. This is subjective, but I’d also trust Core Knowledge with their reputation for integrity, as opposed to Reading First programs that were not only under an ethical cloud, but they also don’t pass the smell test. As Core Knoweldge has shown, if you push tutorials and drills without knowledge, it makes sense that you will produce students who can decode but can’t read for comprehension.

I’m no reading expert, but common sense has served me well as a teacher. Common sense indicates that those politicized efforts of reading instruction, as well as measuring results, will always be undercut by their compromised origins.

By: noble hart

noble hart — Tue, 19 Aug 2008 16:53:57 +0000

For 10 years, Core Knowledge has asked for randomized experimental trials, and any other kinds of trials ed researchers might wish to attempt. No established ed researchers have been willing to pursue this. Have any of you an explanation for that?

There is a good deal of archival data showing large effect sizes of CK. (See their web site). While such studies do not pass the experimental gold standard, these large effect sizes should at least activate the research community. Why not? Has anyone an explanation for that?

And while some programs may have shown highly validated improvement according to some standard, does that count for much if the effect sizes are small? Has anyone an explanation for these small effects?

Has SFA, praised above, in fact greatly enhanced comprehension? If not, has anyone an explanation for that?

By: Cooler heads

Cooler heads — Tue, 19 Aug 2008 15:09:04 +0000

I’m with spj on this. Success for All has allowed its curriculum to be tested in randomized, experimental trials. Core Knowledge has not. Until I see something beyond hype and promises, something that offers evidence of its effectiveness in a randomized trial, then I have to dismiss all the effusive praise of this program.

By: spj

spj — Tue, 19 Aug 2008 14:55:49 +0000

core knowledge ought to subject itself to a long term random assignment trial (5+ years) to see if it actually works, in fact all curriculums ought to be subjected to such tests (kind of like fda for curriculum) in a test phase and if they prove to work, then the feds should allow them on the market, if not, too bad.