High-Stakes Testing vs. Bottom-up Education

Aaron Swartz has an imaginary socratic dialogue with Matt Yglesias about high-stakes testing:

We once allowed each teacher to direct their classroom in their own way, but high-stakes tests and “value-added” measurements now force all of them into the same mold.

Isn’t this a good thing, demands Matt Yglesias? We have science that shows good teaching can make a huge difference in people’s lives—doesn’t everyone deserve the benefits that come from having a good teacher? He dismisses the stories of the individual horrors that result from this process as mere anecdote—inevitably in imposing a one-size-fits-all solution there will be some negative side effects for a few, but the benefits for the many outweigh the costs. Again, I have tried to put this position in its most favorable light (I hope Matt will correct me if I’ve failed) but I’m flabbergasted by its callous naiveté. The problem with allowing hard incentive systems to squeeze out individual judgment is inevitably that people begin trying to game the system—they cheat on the tests, they coach students on the answers, they cut recess and art for more drill-and-skill. To dismiss the on-the-ground evidence of how badly these tests hurt kids, in favor of some Olympian view of the benefits of rising test scores, is ludicrous when the on-the-ground view is telling you the test scores are actually bogus.

Fine, Matt says, that just means we need to crack down on cheating. (This is always the first response of the incentive designer—we just need to improve the incentive system!) The fact that a couple teachers cheat on their students’ tests is no reason to give up on all the benefits better teachers can being. And that’s true, but blatant cheating is just the tip of the iceberg.

I think this is a great point, and it echoes some of the themes I’ve written about before. The testing regime that was the centerpiece of George W. Bush’s No Child Left Behind is a classic example of what James Scott calls high modernism. And it has the same flaws as high modernist projects everywhere: assuming the characteristics being measured are the only ones that matter, and ignoring the costs of the standardization required to do the measuring.

Aaron’s critique of current policies is spot-on, but his suggested alternatives are unsatisfying. He suggests that we “measure students by real results, rather than artificial tests.” And he suggests that we “put [our] trust in teachers” to serve their students. These suggestions aren’t wrong so much as they’re at the wrong level of abstraction. If I had kids, I’d gladly put them in a classroom run by Aaron according to Swartzian pedagogical principles. But as far as I know Aaron isn’t going to become a school teacher, and in any event there’s only one of him.

No Child Left Behind was trying to address a real problem: there really were (and are) schools and teachers who were failing to educate the children entrusted to them. President Bush’s solution was a bad one, as my colleagues at Cato have been arguing for close to a decade. But the problems with our schools long predate the high-stakes testing fad, and they will remain after we do away with them. The question isn’t how to run an individual classroom or school—we have plenty of examples of effective teachers. The question is how to replicate their successes at scale.

My own view is that the fundamental problem is that our education system is organized around a system of geographically-based monopolies. Traditionally, all non-wealthy children within a given district are sent to a school assigned by the local school bureaucracy. This kind of system is dramatically inegalitarian, as the worst schools tend to be in the poorest neighborhoods. And it leaves relatively little room for the kinds of radical pedagogical experimentation Aaron favors; only children with relatively wealthy parents have the option to choose non-traditional methods of instruction.

This is one reason I think Aaron’s hostility to charter schools is misplaced. Charter schools and high-stakes testing are largely advocated by the same people, and high-stakes testing is often part of the charter school evaluation process. But they’re conceptually distinct policies, and it’s perfectly possible to have charter schools that don’t rely on high-stakes testing. And whatever else you might say about charter schools, they’re certainly more amenable to bottom-up experimentation than traditional public school bureaucracies—especially in large urban districts that are most in need of reform.

Unfortunately, thanks to peculiarities of American political culture, decentralizing the provision of education has come to be regarded as a right-wing idea in the United States. But there’s nothing particularly conservative about making it easier for a guy like Aaron Swartz to create a school organized on dramatically different principles than existing public schools, and for non-wealthy parents to send their kids there. Insights about improving the performance of individual schools is important, but not as important as solving the meta-problem of making the education system as a whole more hospitable to experimentation and entrepreneurship.