Aaron Swartz has an imaginary socratic dialogue with Matt Yglesias about high-stakes testing:
We once allowed each teacher to direct their classroom in their own way, but high-stakes tests and “value-added” measurements now force all of them into the same mold.
Isn’t this a good thing, demands Matt Yglesias? We have science that shows good teaching can make a huge difference in people’s lives—doesn’t everyone deserve the benefits that come from having a good teacher? He dismisses the stories of the individual horrors that result from this process as mere anecdote—inevitably in imposing a one-size-fits-all solution there will be some negative side effects for a few, but the benefits for the many outweigh the costs. Again, I have tried to put this position in its most favorable light (I hope Matt will correct me if I’ve failed) but I’m flabbergasted by its callous naiveté. The problem with allowing hard incentive systems to squeeze out individual judgment is inevitably that people begin trying to game the system—they cheat on the tests, they coach students on the answers, they cut recess and art for more drill-and-skill. To dismiss the on-the-ground evidence of how badly these tests hurt kids, in favor of some Olympian view of the benefits of rising test scores, is ludicrous when the on-the-ground view is telling you the test scores are actually bogus.
Fine, Matt says, that just means we need to crack down on cheating. (This is always the first response of the incentive designer—we just need to improve the incentive system!) The fact that a couple teachers cheat on their students’ tests is no reason to give up on all the benefits better teachers can being. And that’s true, but blatant cheating is just the tip of the iceberg.
I think this is a great point, and it echoes some of the themes I’ve written about before. The testing regime that was the centerpiece of George W. Bush’s No Child Left Behind is a classic example of what James Scott calls high modernism. And it has the same flaws as high modernist projects everywhere: assuming the characteristics being measured are the only ones that matter, and ignoring the costs of the standardization required to do the measuring.
Aaron’s critique of current policies is spot-on, but his suggested alternatives are unsatisfying. He suggests that we “measure students by real results, rather than artificial tests.” And he suggests that we “put [our] trust in teachers” to serve their students. These suggestions aren’t wrong so much as they’re at the wrong level of abstraction. If I had kids, I’d gladly put them in a classroom run by Aaron according to Swartzian pedagogical principles. But as far as I know Aaron isn’t going to become a school teacher, and in any event there’s only one of him.
No Child Left Behind was trying to address a real problem: there really were (and are) schools and teachers who were failing to educate the children entrusted to them. President Bush’s solution was a bad one, as my colleagues at Cato have been arguing for close to a decade. But the problems with our schools long predate the high-stakes testing fad, and they will remain after we do away with them. The question isn’t how to run an individual classroom or school—we have plenty of examples of effective teachers. The question is how to replicate their successes at scale.
My own view is that the fundamental problem is that our education system is organized around a system of geographically-based monopolies. Traditionally, all non-wealthy children within a given district are sent to a school assigned by the local school bureaucracy. This kind of system is dramatically inegalitarian, as the worst schools tend to be in the poorest neighborhoods. And it leaves relatively little room for the kinds of radical pedagogical experimentation Aaron favors; only children with relatively wealthy parents have the option to choose non-traditional methods of instruction.
This is one reason I think Aaron’s hostility to charter schools is misplaced. Charter schools and high-stakes testing are largely advocated by the same people, and high-stakes testing is often part of the charter school evaluation process. But they’re conceptually distinct policies, and it’s perfectly possible to have charter schools that don’t rely on high-stakes testing. And whatever else you might say about charter schools, they’re certainly more amenable to bottom-up experimentation than traditional public school bureaucracies—especially in large urban districts that are most in need of reform.
Unfortunately, thanks to peculiarities of American political culture, decentralizing the provision of education has come to be regarded as a right-wing idea in the United States. But there’s nothing particularly conservative about making it easier for a guy like Aaron Swartz to create a school organized on dramatically different principles than existing public schools, and for non-wealthy parents to send their kids there. Insights about improving the performance of individual schools is important, but not as important as solving the meta-problem of making the education system as a whole more hospitable to experimentation and entrepreneurship.
This doesn’t go nearly far enough, unfortunately. It’s institutionalized education itself that is a stultifying exercise in high modernism. I find it utterly confusing that, in 2011, with all the world’s information available (essentially) for free, we still send children off to prisons where they can be instructed by people who all bomb the GRE. Fund some really impressive public libraries and let kids use or not use them as they see fit.
Interesting constructive reply. Good point about high modernism. However…
I don’t think parents are well equipped to make judgements about school quality, especially lacking relevant metrics. So an approach that relies on competition with no metrics could lead to the “school market” becoming something like the “breakfast food market” which I hope we can agree we wouldn’t want to emulate.
At a minimum we need to establish metrics that parents (and funders, and accreditation organizations, etc.) can use in their judgements. I don’t know the literature, but basic content focused multiple choice testing testing seems extremely unlikely to be a good choice. Providing that to parents probably wouldn’t help much, it could even make things worse.
Ideally we’d be looking at outcomes: After 20 years, how happy, healthy, and socially valuable are entrants to the school, compared with peers matched by socio-economic status, intelligence, height, race, etc.? But a metric that can’t be computed for 20 years has some obvious problems.
So we need to find out what metrics we can generate quickly that correlate well with the long term metrics. My guess is that metrics like that wouldn’t offer any simple way to “teach to the test”.
Also, testing every student simply isn’t needed to measure the school; it is wasteful. Valid metrics can be generated by testing a relatively small randomly selected sample of students. If we’re only testing samples, metrics could involve relatively expensive testing or even, for example, functional MRI scans.
And of course separating the testing process from the school administration and environment would largely resolve existing cheating opportunities. Furthermore it would make it easy to test home-schooled or unschooled children on an equal footing. This would give us a chance to get out of the box that Rowz describes so well.
My guess is that the reason we don’t even consider this sort of path is that it doesn’t at all align with the institutional imperatives of the existing system. But that pretty much shows what’s wrong with the existing system.
This is a very interesting post. (I got here via E.D. Kain’s blog.) But you speak of the charter school movement as a monolith. It’s not. A community-based, educator- or Aaron Swartz-developed charter school is one thing, but when you get charter school organizations and companies getting in the mix (for-profit or non-profit), then eventually you’ll get the same stuff as before, but in charter school form, with very little oversight–you get a shadow top-down bureaucracies and monopolies, not to mention franchises. Besides who or what is behind a charter school, it’s also important to look at the how. Each state has different laws regulating the how and in some cases the who.
I partially agree with Rachel, that how the charter schools come to be can be important. I suspect I disagree with her on a couple other crucial thing:
(a) Franchising isn’t all bad. I wouldn’t want the only options to be charter schools offered by outside organizations, but I think their inclusion in the mix is a good thing.
(b) The “very little oversight” actually benefits Swartz schools more than it benefits the large, outside organizations. If anything, I would be worried about too much oversight being to the benefit of the latter by creating all sorts of hoops that it’s hard for five teachers with a dream to meet but easy for a corporation like Edison to meet.
More broadly, charter schools can’t be code for “schools I want to see”, because different people want to see different things. Personally, I want to see a little bit of everything. But there are some things I am not very interested in seeing at all an types of schools that I would never send my kid to. But some other parent might, and I’d be happy to tolerate it, provided that they met some pretty minimum standard and parents got to choose whether their kids would go there or not.
Trumwill,
With a few exceptions, on charter schools there’s only so vehemently I can agree or disagree because I’m still figuring out how it all works. I think you and I would also have to have a longer conversation clarifying what we each mean by “oversight.” If by “oversight” you mean, overbearing or stifling rules and regulations/ bureaucracy, I’d agree that’s problematic. But to me “oversight” can mean making sure that part of the five teachers’ dream (or implementation thereof) isn’t teaching US History according to David Barton–this, in a public school, I vehemently disagree with. Oversight can also mean making sure that charter school administrators aren’t awarding themselves over-sized salaries while their teachers are left engaging in innovations such as limiting the learning process of their students to filling out worksheets.
My main point, though, was that we have to be careful not to recreate or add more stifling, top-down bureaucracies as we’re running from them.
PS – I very much appreciate the conversation.
But to me “oversight” can mean making sure that part of the five teachers’ dream (or implementation thereof) isn’t teaching US History according to David Barton–this, in a public school, I vehemently disagree with.
I’m pretty flexible on learning methods (more on that in a minute), but much less so on actual curriculum. I have to say, though, that my being inflexible is an inherently top-down perspective. It’s an area in which I frequently find myself in conflict with bottom-up people. (I’m somewhere in between)
Oversight can also mean making sure that charter school administrators aren’t awarding themselves over-sized salaries while their teachers are left engaging in innovations such as limiting the learning process of their students to filling out worksheets.
It’s pretty doubtful that I would want to send my kid to such a school, but (as mentioned above) I am pretty flexible on letting schools try different things. While the worksheet-all-day model sounds particularly odious to me, there are ideas that EDK finds extremely objectionable that I would like to see tried out. No objection to keeping an eye on profit margins and administrative salaries, though, provided that we’re giving them some flexibility.
Interesting post and I agree with your argument against testing, however your defense of charter schools leaves out one important component: Involved parents are vital to having successful schools. This is why the geographic model is so appealing – because parent who live closer to a school are more likely to be more involved. No scientific study here, just my observations from my experience. So, if you are going to talk about quality of schools or charter schools, you have to talk about parental involvement too or else you are just going to be looking for another way (whether testing or the market or whatever) to evaluate teacher performance when the best way to make sure we have good teachers is to have parents involved.
I remember the left wing decentralization experiment in NYC back in 1968. It led to a great deal of corruption as the neighborhood organizations became centers of patronage. In fact, it was the teacher’s union that fought off attempts to turn teacher hiring into a political spoils system for disbursement by local board politicians. (There was a big strike over this.) Needless to say, it didn’t improve the schools and has largely been undone over the past four decades.
As best I can tell, the right wing decentralization experiment is producing similar results. Yes, there are charter schools that are doing a good job educating children, but an awful lot of them are simply contractor boondoggles providing good executive and consultant salaries at the expense of the taxpayers, teachers and school children. Getting rid of a charter school is even harder than firing a teacher. Corporations have better lawyers than the teacher’s unions, and they have better lobbyists.
Be careful what you wish for.