Bottom-up

Jim Henley makes a good observation:

Across a whole range of problems there’s a class of responses I’ll dub the “low road” and another class I’ll call the “high road.” Examples of the former include war, torture, sanctions and blockades, imprisonment, aversive conditioning of all types (spanking; “dominance”-based animal training). Examples of the latter include diplomacy, rapport-building, civil disobedience, the free exchange of goods and ideas, decriminalization and rehabilitation, positive conditioning (of humans and animals).

I don’t presently care to argue that there is never any “need” to go down any given low road. In some cases I may support some low roads for some purposes. Locking up murderers, for instance. In other cases – torture – I have a much easier time saying “Never go there.” But what we see over and over again is that we judge high-road approaches as failures unless they produce nigh-instant and complete favorable results, while we show nearly infinite patience for journeys down the low road.

Nine years into the invasion of Afghanistan we have to agree that pulling out after a decade is just too soon. Back in 2001, the Taliban’s failure to turn over Osama bin Laden within a couple of weeks showed the hopelessness of diplomacy. When torture “works” at all it takes weeks and months, just like more classic rapport-building methods of interrogation. And it involves more false positives. Plus, oh I forgot to mention, it is deeply evil. But even though classic interrogation methods produce statistically better results, we live in fear that there may be some time somewhere that torture might get an answer that classic interrogation missed, so of course we must continually torture for that possible moment’s sake…

The war on drugs will surely work at some point – we’ve only been at it for 90-odd years, trillions of dollars and countless deaths and humiliations. But should anyone anywhere decriminalize anything, a single death or inconvenience in the first week would condemn the entire effort.

The distinction Jim’s drawing here is closely related to the top-down/bottom-up distinction I’ve been exploring. A big part of the problem, I think, is that it’s just much harder to articulate how and why “high road”, bottom-up strategies will work.

We’ve seen how counterintuitive people find Wikipedia compared with the traditional editing process. In Jim’s examples, the same basic fallacy is at work, there’s just more violence involved. It’s easy to tell a story about how a blockade will work: you make a list of stuff you don’t want the regime you’re blockading to have and prevent it from having them. It’s much harder to explain how free trade can undermine despotic regimes, because it’s a decentralized process involving thousands of personal cross-border relationships. Likewise, it’s easy to explain how shooting a suspected terrorist with a Predator will reduce future terrorism. It’s much harder to explain the subtle ways that declining to shoot a suspected terrorist (who might, remember, actually be innocent) with a Predator drone might help in the long-run effort to fight terrorists. So too often we opt for top-down strategies that produce what seem to be direct and immediate results (even though they have terrible side effects) over bottom-up strategies that work in more subtle and indirect ways but work better in the long run.

Bilski v. Kappos was the most-anticipated Supreme Court patent case in a generation. And when it was finally handed down on Monday, it turned out to be the most anticlimactic.

There’s been a raging debate about software and “business method” patents since an appeals court gave the green light to them in 1998. Many people, myself included, hoped that the Supreme Court would place new limits on such patents this term.

But Bilski turned out to be a bad test case. The applicant, one Bernard L. Bilski, tried to patent a “method for managing the consumption risk costs of a commodity.” If that doesn’t sound like the sort of thing patents are supposed to cover, that’s because it’s not. Almost no one other than Bilski and his attorney believed that he should get his patent.

So there was little doubt that Bilski would lose, what mattered was how. A narrow ruling would simply invalidate Bilski’s particular patent but leave the bulk of business method (and software) patents intact. A broad ruling, in contrast, could dramatically limit such patents.

As it turned out, the Supreme Court was split down the middle. The court’s liberals sided with an opinion by Justice Stevens, while four of the court’s conservatives backing an opinion by Justice Kennedy. That might not seem surprising on today’s polarized Supreme Court, but it’s a new development for patent law. On the rare occasions when the justices have disagreed about a patent case, (as they did with dueling concurrences in eBay v. MercExchange) there hasn’t been a clear partisan split.

But in this case, the conservatives favored the status quo, while the liberals wanted to place new limits on business method patents. Bilski was argued in November, so a decision was expected in March or April; court watchers were puzzled when June arrived with no decision. My guess is that the swing voter, Justice Scalia, spent those months negotiating to decide which opinion to join. Ultimately, Scalia sided with his fellow conservatives, giving Justice Kennedy a 5-vote majority.

Yet Scalia didn’t join the full Kennedy opinion. He opted out of passages like this one:

The machine-or-transformation test may well provide a sufficient basis for evaluating processes similar to those in the Industrial Age—for example, inventions grounded in a physical or other tangible form. But there are reasons to doubt whether the test should be the sole criterion for determining the patentability of inventions in the Information Age. As numerous amicus briefs argue, the machine-or-transformation test would create uncertainty as to the patentability of software, advanced diagnostic medicine techniques, and inventions based on linear programming, data compression, and the manipulation of digital signals.

Seeing this in a majority opinion would be pretty depressing for a software patent critic like me, but because Justice Scalia didn’t sign this section, it’s not controlling precedent. Back in 2007 I noticed that Justice Scalia (along with Stevens and Breyer) was asking questions that suggested skepticism about the patentability of software. Scalia’s position in Bilski confirms that impression: Whenever Justice Kennedy waxes poetic about the Information Age, Justice Scalia gets off the bus. The result is an exceptionally narrow holding that doesn’t give much comfort to partisans on either side.

The primary winners from all this are the patent lawyers. Not only are there few restrictions on what can be patented, but the high court’s failure to articulate a clear rule means even more litigation. All is not lost, though. Justice Stevens is retiring, but Justices Breyer, Ginsburg, and Sotomayor all signed onto his opinion. If Justice Kagan sides with her fellow liberals, then they’ll need just one more vote to restore some sanity to patent law. As the patent system continues to inflict damage on the IT industry, the need for reform will only get more obvious. Hopefully, the next time the Supreme Court has an opportunity to fix the problem, Justice Scalia will get off the fence.

Over at Ars Technica, I’ve got an article examining the tech policy record of Justice John Paul Stevens, who retired today:

Stevens penned the 1978 decision that shielded the software industry from the patent system in its formative years. In 1984, Hollywood’s effort to ban the VCR failed by just one Supreme Court vote; Stevens wrote the majority opinion. And in 1997, he wrote the majority opinion striking down the worst provisions of the Communications Decency Act and ensuring that the Internet would have robust First Amendment protections.

Indeed, Justice Stevens probably deserves more credit than any other justice for the innovations of the last three decades. And given how central those technologies have become to the American economy, Stevens’ tech policy work may prove one of his most enduring legacies. In this feature, we review Justice Stevens’s tech policy decisions and salute the justice who helped make possible DRM-free media devices, uncensored Internet connections, free software, and much more.

A couple of follow-ups on my post about the decline of reporter impartiality. First, Jim Henley points out that the ethos of the opinionated blogger isn’t really new:

I submit that this is just magazine-journalism ethos with the addition of cat pictures. If you think about what good long and short-form journalism looks like at a decent magazine, it looks like the bullet-points above. I’m not just talking about ideological organs. The writer who sells to Harper’s or The National Geographic or even Runner’s World is going to tend to show a personality and take a definite perspective, while at the same time doing fresh reporting from primary sources, whether human, documentary or physical. The writer will make sure to include a substantial account of challenges to her perspective, if only to knock it down later.

If you were a magazine editor and knew Paul Theroux hated the English because he wrote an entire book about how much the English suck, you might still send him to write a big piece on England for your monthly because you expected it to be interesting, and because the ethos of magazine journalism would make it “fair.” If you knew that William Greider hated economic conservatives, or Tom Wolfe hated social liberals, you would still buy factual pieces touching economics from the one and cultural folkways from the other: their very names constitute warning labels; their strong viewpoints sharpen their writing; and because they’re professionals, they’ll put in the work.

Second, Matt Yglesias speculates on the long-term trajectory of the Post:

The odds are quite good that The Washington Post won’t exist in anything remotely resembling its current form in 20 or 30 years. It’s possible that some local news outlet buy the trademark, but realistically a digital world only needs so many general purpose English language news sources and there are many better-positioned brands and firms out there. The BBC, the New York Times, the Associated Press, and Reuters as is News Corporation’s family of brands. Beyond that, Time Warner’s family brands has considerable strength, so does the rapidly growing NPR…

Beyond those seven I named of course you might see other survivors. But it’s going to be tough out there. Really tough. And in some ways it’s especially tough for an organization like The Washington Post. As you can read here, a large number of Post staffers loathe and despise both Weigel and Ezra Klein, the paper’s two signature efforts to obtain relevance in a digital age. In part that’s small-minded of them, but in large part it’s inevitable. Naturally the ethos cultivated at mid-sized urban daily is different from the ethos of the new media. That makes it hard to dip your toes into the new waters. But that in turn makes it extremely difficult to compete with smaller, nimbler organizations that can more easily succeed at opportunistically snagging talented people and not worrying so much about how it fits into the larger scheme of things.

This seems spot-on. I made a similar point last year.

On Friday, the DC media world was abuzz with the news that Dave Weigel resigned from the Washington Post after intemperate comments he’d made on a private mailing list were made public. Dave’s a friend of mine, so unsurprisingly I think the Post screwed up. But plenty of people have made that case better than I could, so I’ll just endorse Marc Ambinder and Julian Sanchez‘s takes.

Instead, I want to talk a bit about the structural factors that led to Dave and the Post parting ways on Friday. There’s a huge cultural chasm between Dave’s defenders and his critics. On the one hand are folks who argue that the Post was simply defending traditional principles of journalistic objectivity. For example, the Post‘s ombudsman says that “Weigel’s e-mails showed strikingly poor judgment and revealed a bias that only underscored existing complaints from conservatives that he couldn’t impartially cover them.”

Dave’s defenders, on the other hand, believe that the notion of reporter impartiality has always been something of a myth. Reporters inevitably form opinions about the subjects they cover, and members of Team Weigel argue that the pretense of objectivity has always been just that: a pretense. On this view, it’s better to have reporters be candid with their readers about their own biases, and then readers can take that into account in judging their work.

The tendency to debate the issue as an abstract question of journalistic ethics misses what’s actually going on here. The ethos of journalistic objectivity isn’t declining because its opponents suddenly came up with stronger arguments. Rather, the shift in journalistic behavior is driven by economic forces that are making Weigel-style journalism more profitable and Post-style journalism less so.

To understand why, we have to understand where the ethic of journalistic objectivity came from in the first place. Because of the tremendous economies of scale in newspaper printing and distribution, operating a high-circulation paper is much more profitable than operating a low-circulation one. A smaller paper has to devote a larger fraction of its budget to basics like printing and distribution. That leaves less money for hiring reporters, which harms the quality of the paper’s content. And inferior content makes it hard to attract subscribers.

The Washington Star was the last serious Post competitor to succumb to this vicious cycle in 1981. It was quickly replaced by the Washington Times, which has only been able to stay in business because Sun Myung Moon was prepared to cover the paper’s losses in perpetuity in order to promote his conservative views. In its heyday circa 1990, the Washington Post had something like 800,000 subscribers, an order of magnitude more than any other DC daily. There are fewer than 2 million households in DC, so by this point the Post was operating in a relatively saturated market: not every household was a Post subscriber, but most of those that were interested in taking a newspaper were taking the Post.

In a winner-take all market like this, it pays to be seen as scrupulously neutral and even-handed. Being seen as “the liberal paper” effectively concedes half of the market to a potential competitor with few offsetting benefits. A reporter that inspired fanatical loyalty among 10 percent of the population but angered a different 10 percent would have been bad for the bottom line: The loyal 10 percent would probably have subscribed anyway, but the angry 10 percent might cancel their subscriptions out of spite if a particular reporter angers them enough. So the most successful newspapers tended to be the ones whose reporters pretended not to have opinions. And as papers with that culture came to dominate the industry, it came to be seen as not just good business strategy but as central to Journalistic Ethics.

The Internet fundamentally changes the economics of the news business. Running 100 newspapers with 5 reporters each is much more expensive than running a single newspaper with 500 reporters, so the winning business strategy in 1990 was to get as big as possible. On the web, distribution costs are trivial, which means that running 100 online news outlets with 5 reporters each (like Dave’s former perch at the Washington Independent) costs about the same as running a single online news outlet with 500 reporters. If washingtonpost.com needs (say) 10 million daily visitors to pay the salaries of its 500 reporters, then washingtonindependent.com, can support its 5 reporters with around 100,000 daily readers.

A polarizing reporter can be an asset for a small publication. The Windy isn’t expecting to dominate the market, so it’s a perfectly viable business strategy to cater to left-leaning readers and ignore right-leaning ones. If one of its reporters inspires fanatical loyalty among left-wing readers and venomous hatred among right-wingers, that’s not a problem; the lefties will stick around and the conservatives probably weren’t going to be regular readers anyway.

There are now dozens of sites pursuing this strategy: building small, edgy, opinionated news outlets with a handful of reporters catering to a relatively narrow audience. It turns out that when given a choice, a lot of people prefer this style of reporting to to the bland, one-size-fits-all stuff the Post is serving up. Individually, none of these sites poses a threat to major outlets like the Post but collectively they’re causing a death of a thousand cuts, as each segment of their audience is bled away by a different, tiny competitor.

Things have gotten so grim that the paper is flailing, trying one strategy after another in hopes of hitting on a strategy that will work. Hiring Dave and Ezra Klein was apparently one such attempt. Unfortunately, the paper didn’t do its homework. They seem not to have read enough of Dave’s past work to realize that his style of reporting was fundamentally different from the style practiced by other reporters at the Post. When they finally realized this fact, they rather ludicrously blamed Dave for his failure to be “impartial,” despite the fact that Dave has never pretended to be.

There’s a tendency among professional blogger types to cluck their tongues and say that the Post just needs to start behaving in a more bloggy fashion and everything will be OK. But I think that largely misses the point. Hiring opinionated reporters can be a good strategy in general, but it’s probably not a good strategy for the Washington Post. A large, bureaucratic organization like the Post is almost certainly incapable of nurturing the kind of quirky, bottom-up culture you that produces successful bloggers. And the business strategy of the Post requires that it appeal to a broad audience. You don’t do that by hiring some bloggers who offend liberals and other bloggers who offend conservatives; that will just alienate everyone.

The decline of the ethic of journalistic impartiality is just one facet of the larger decline of cellulose-based information technologies that are the foundation of the Washington Post‘s business. Late-20th-century print journalists fooled themselves into thinking that the journalistic culture of large daily newspapers was the gold standard for journalists in general. In reality, each medium has its own distinctive style. The Internet is still a young medium, and so it’s too early to say what the new culture will look like. But the key point is that the optimal style for online journalism is something that will be discovered by trial and error. Abstract arguments about journalistic ethics are sort of beside the point.

The more I think about it the more trouble I’m having understanding what people mean when they say they’re against [Edit: illegal] immigration. Analogies are never perfect, but consider driving. I speed on a semi-regular basis. And virtually everyone else on the road seems to do so as well. It’s not only excused but almost expected: someone who drives exactly the speed limit when everyone else is going 15 over is more likely to get dirty looks than to be admired for his respect for the law.

Speeding is illegal. You can get fined for it, and if you do it fast enough and often enough you can lose your license and even go to jail. Moreover, speeding kills.

Yet we don’t have a national debate about “illegal driving.” No one frets that peoples tendency to drive over the speed limit threatens the rule of law. People would think you were crazy if you said: “I don’t have a problem with driving, but it needs to be done legally.” Nobody complains that tolerating people who go 65 in a 55 zone is unfair to other drivers who are driving exactly 55. Bill O’Reilly and Lou Dobbs doesn’t do segments about “illegals” (that is, people who engage in “illegal driving”) and what a menace to society they are.

Rather, people have the sensible view that it’s not a big deal if people break the law when doing so makes sense in context. As I’ve pointed out before, there’s a long list of laws that people we would otherwise regard as law-abiding flout on a regular basis. Nobody thinks the rule of law is imperiled because people jaywalk or fail to pay their use taxes.

Being in the United States without the proper documentation strikes me as being in this same class of offenses. It’s a classic paperwork violation; by itself it harms no one. Yet for reasons that aren’t clear to me, millions of people who don’t think twice about driving 70 in a 55 zone go absolutely berserk when it’s suggested that maybe we should forgive a smart, hard-working kid whose parents didn’t have the right paperwork 15 years ago.

Of course, some illegal immigrants do things that impose costs on others: the commit crimes, go on welfare, demand free medical care, and so forth. But if that’s the concern, then that’s what we should be cracking down on. More to the point, if that’s what you’re concerned with, you certainly should support creating a path to citizenship for kids like Balderas who’s done everything right and will almost certainly be a law-abiding citizen and a net taxpayer.

So what’s going on here? Why are people who casually engage in “illegal driving,” “illegal Internet shopping,” “illegal street crossing,” and so forth so obsessed with stopping “illegal immigration?” Would be interested to read peoples’ thoughts in the comments.

This Rolling Stone profile of General McChrystal has recently led to his downfall. But the really troubling thing about the article isn’t what McChrystal thinks of his civilian superiors (although that is troubling) but the picture it paints of the war effort itself:

One soldier shows me the list of new regulations the platoon was given. “Patrol only in areas that you are reasonably certain that you will not have to defend yourselves with lethal force,” the laminated card reads. For a soldier who has traveled halfway around the world to fight, that’s like telling a cop he should only patrol in areas where he knows he won’t have to make arrests. “Does that make any fucking sense?” asks Pfc. Jared Pautsch. “We should just drop a fucking bomb on this place. You sit and ask yourself: What are we doing here?”

The rules handed out here are not what McChrystal intended – they’ve been distorted as they passed through the chain of command – but knowing that does nothing to lessen the anger of troops on the ground. “Fuck, when I came over here and heard that McChrystal was in charge, I thought we would get our fucking gun on,” says Hicks, who has served three tours of combat. “I get COIN. I get all that. McChrystal comes here, explains it, it makes sense. But then he goes away on his bird, and by the time his directives get passed down to us through Big Army, they’re all fucked up – either because somebody is trying to cover their ass, or because they just don’t understand it themselves. But we’re fucking losing this thing.”

The fundamental problem here is the mismatch between the top-down strategy that Gen. McChrystal sold to Pres. Obama and the way that strategy has translated into the orders given to men on the ground. It’s hard to know from the outside whether the problem is that the orders got garbled as they went down the chain of command, or if the orders were incoherent to start with. But in a sense it doesn’t matter: if your strategy is too complex or subtle to be implemented correctly by Big Army, then it’s not a good strategy no matter how good it might have looked on a PowerPoint slide.

Pres. Obama has announced that Gen. Patraeus will be taking over for Gen. McChrystal, and he says that “This is a change in personnel but it is not a change in policy.” Which is a shame, because the most important lesson I draw from the Rolling Stone article is that the people actually charged with carrying out the policy don’t think it’s working.

I’ve been blogging about immigration over at Megan’s blog. On Friday I called out Mint.com for posting an inflammatory info-graphic on its blog (which the company subsequently pulled). And today I consider the common argument that the rule of law demands that we take a tough line against illegal immigrants.

My friends Megan McArdle and Peter Suderman got married in DC this weekend. For the next couple of weeks I’ll be guest-blogging for Megan while she’s on her honeymoon. Here is a post where I give my impressions of her wedding and ponder our promiscuous use of Twitter at the reception. And here I wade into the wonky but important debate over credit card interchange fees. Megan has lined up a stellar team of guest bloggers so I encourage you to subscribe to her blog if you aren’t already. I’ll also be doing a few posts here, so stay tuned.

The point I made on Friday about the dangers of over-reliance on (possibly faulty) data seems obvious in hindsight. You might think that McNamara just happened to have an unusual blind spot, and that his problems could have been avoided relatively easily by applying an appropriate dose of skepticism.

But the problem is much harder to avoid than it seems at first glance. And it’s hardly limited to the military. Recall that one of the defining characteristics of disruptive technology is that it’s typically inferior to existing technologies if measured by conventional means—that is, according to conventional statistics. In the 1970s, for example, microcomputers were inferior to minicomputers by almost every conventional criteria. Incumbents like DEC analyzed the microcomputer using conventional measures like speed, storage space, reliability, and peripheral support, and concluded that they weren’t serious threats to its business. But of course the traditional statistics, which were geared toward the needs of existing customers, didn’t do a good job of capturing the unique advantages—small size, low power consumption, and most importantly low price—of the microcomputer.

Problems with faulty data tend to sneak up on well-established organizations when the assumptions on which the organization was founded cease to apply. One reason the military was so confident in its statistics in the 1960s is that similar statistics had worked well in the three conventional wars American had fought in the previous half-century. Knowing how many Nazis you’ve killed and how many tons of bombs you’ve dropped on Dresden really were reasonable proxies for how well the war was going overall.

On paper, an Apple II was inferior to a PDP just as on paper, the Vietcong were inferior to the US military. But that’s because the “paper” measurements were measurements of the wrong variables. Distilling a product or an army to a collection of facts and figures drained them of the context that might have allowed a smart commander or business executive to realize that the world had changed.

Large organizations are especially prone to making this kind of error. There are usually people near the bottom of an organization who figure out relatively quickly that the official statistics were wrong. Rank-and-file soldiers knew the war was going poorly, just as many DEC engineers and sales people knew that the microcomputer was a serious competitive threat. Indeed, in The Innovator’s Dilemma Christensen observes that there are usually engineers in incumbent firms lobbying their bosses to let them enter disruptive markets. In some cases, they get as far as building prototypes. In other cases, they get frustrated and leave to start their own companies.

This is a problem that’s particularly severe for larger organizations. Large companies have more intra-bureaucratic politics, and having “hard” numbers at your fingertips is a tremendous advantage in such fights. Typically, it doesn’t matter very much if you have the “right” statistics—even bad statistics are better than nothing. When debates over war strategy occurred inside the Johnson administration, the hawks at the Pentagon had reams of extremely detailed (albeit mostly worthless) data. They could give extremely precise statistics about the (apparent) successes of various military campaigns: number of enemy killed, number of bombs dropped, territory cleared, etc. The doves at the State Department, in contrast, were trying to make arguments that were much subtler. They lacked the military’s resources for collecting data, and their claims wouldn’t have been easy to quantify anyway. So you had a situation where one side of the debate always came prepared with precise statistics and detailed plans, while the other side raised concerns that seemed extremely vague, abstract, and hypothetical. The people with the statistics were usually wrong, but they tended to get their way anyway.

This sort of problem is much less likely to crop up in small organizations. Senior executives is close enough to the rank and file that they’re likely to be familiar with the context surrounding official statistics. A good CEO at a small firm will see and talk to ordinary employees often enough that he’ll catch wind of problems long before they begin showing up in the official statistics. In a very large organization, in contrast, there are several layers of bureaucracy between the guys doing the work and the guys making the decisions. As a consequence, managers often have to rely on statistics stripped of context. And so they continue to believe they have a winning product—or military strategy—right up until the moment that disaster strikes.