How To Destroy Surveillance Capitalism

Cory Doctorow book: How To Destroy Surveillance Capitalism ISBN:1736205900

The net of a thousand lies

The most surprising thing about the rebirth of flat Earthers in the 21st century is just how widespread the evidence against them is

The arguments for ridiculous beliefs in odious conspiracies like anti-vaccination, climate denial, a flat Earth, and eugenics are no better than they were a generation ago. Indeed, they’re worse because they are being pitched to people who have at least a background awareness of the refuting facts. (cf conspiracy theory)

So can these far-fetched conspiracy theorists really be succeeding on the basis of superior arguments? Some people think so. Today, there is a widespread belief that machine learning and commercial surveillance can turn even the most fumble-tongued conspiracy theorist into a svengali who can warp your perceptions and win your belief by locating vulnerable people and then pitching them with A.I.-refined arguments that bypass their rational faculties and turn everyday people into flat Earthers, anti-vaxxers, or even Nazis.

But what if there’s another explanation? What if it’s the material circumstances, and not the arguments, that are making the difference for these conspiracy pitchmen? What if the trauma of living through real conspiracies all around us — conspiracies among wealthy people, their lobbyists, and lawmakers to bury inconvenient facts and evidence of wrongdoing (these conspiracies are commonly known as “corruption”) — is making people vulnerable to conspiracy theories? (cf BigWorld)

firefighting is reactive. We need fire prevention . We need to strike at the traumatic material conditions that make people vulnerable to the contagion of conspiracy

Figuring out what we want our tech to look like is crucial if we’re going to get out of this mess. Today, we’re at a crossroads where we’re trying to figure out if we want to fix the Big Tech companies that dominate our internet or if we want to fix the internet itself by unshackling it from Big Tech’s stranglehold. We can’t do both, so we have to choose.

Digital rights activism, a quarter-century on

Digital rights activism is more than 30 years old now. The Electronic Frontier Foundation turned 30 this year; the Free Software Foundation launched in 1985

digital rights activism is right where it’s always been: looking out for the humans in a world where tech is inexorably taking over.

The latest version of this critique comes in the form of “surveillance capitalism,” a term coined by business professor Shoshana Zuboff in her long and influential 2019 book, The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power . Zuboff argues that “surveillance capitalism” is a unique creature of the tech industry and that it is unlike any other abusive commercial practice in history

She’s right that capitalism today threatens our species, and she’s right that tech poses unique challenges to our species and civilization, but she’s really wrong about how tech is different and why it threatens our species

What’s more, I think that her incorrect diagnosis will lead us down a path that ends up making Big Tech stronger, not weaker. We need to take down Big Tech, and to do that, we need to start by correctly identifying the problem

Tech exceptionalism, then and now

Around the turn of the millennium, serious people ridiculed any claim that tech policy mattered in the “real world.”

The “surveillance capitalism” critique recasts the digital rights movement in a new light again: not as alarmists who overestimate the importance of their shiny toys nor as shills for big tech but as serene deck-chair rearrangers whose long-standing activism is a liability because it makes them incapable of perceiving novel threats as they continue to fight the last century’s tech battles.

Don’t believe the hype

Big Tech’s customers are advertisers, and what companies like Google and Facebook sell is their ability to convince you to buy stuff. Big Tech’s product is persuasion.

The fear of surveillance capitalism starts from the (correct) presumption that everything Big Tech says about itself is probably a lie. But the surveillance capitalism critique makes an exception for the claims Big Tech makes in its sales literature — the breathless hype in the pitches to potential advertisers

Surveillance capitalism assumes that because advertisers buy a lot of what Big Tech is selling, Big Tech must be selling something real

Being watched changes your behavior, and not for the better

But Zuboff also claims that surveillance literally robs us of our free will — that when our personal data is mixed with machine learning, it creates a system of persuasion so devastating that we are helpless before it

What is persuasion?

To understand why you shouldn’t worry about mind-control rays — but why you should worry about surveillance and Big Tech — we must start by unpacking what we mean by “persuasion.”

the predictions that surveillance capitalism delivers to its customers are much less impressive

Rather than finding ways to bypass our rational faculties, surveillance capitalists like Mark Zuckerberg mostly do one or more of three things:

1. Segmenting

Surveillance capitalism is segmenting times a billion. Diaper vendors can go way beyond people in maternity wards

This is seriously creepy. But it’s not mind control*

Think of how surveillance capitalism works in politics. Surveillance capitalist companies sell political operatives the power to locate people who might be receptive to their pitch

Because targeting improves the yields on political pitches, it can accelerate the pace of political upheaval by making it possible for everyone who has secretly wished for the toppling of an autocrat — or just an 11-term incumbent politician — to find everyone else who feels the same way at very low cost. This has been critical to the rapid crystallization of recent political movements including Black Lives Matter and Occupy Wall Street as well as less savory players like the far-right white nationalist movements that marched in Charlottesville.

It’s important to differentiate this kind of political organizing from influence campaigns; finding people who secretly agree with you isn’t the same as convincing people to agree with you.

2. Deception

Lies and fraud are pernicious, and surveillance capitalism supercharges them through targeting. If you want to sell a fraudulent payday loan or subprime mortgage, surveillance capitalism can help you find people who are both desperate and unsophisticated and thus receptive to your pitch.

3. Domination

Monopoly is the cause, and surveillance capitalism and its negative outcomes are the effects of monopoly.

One example of how monopolism aids in persuasion is through dominance: Google makes editorial decisions about its algorithms that determine the sort order of the responses to our queries

Google’s dominance over search — more than 86% of web searches are performed through Google — means that the way it orders its search results has an outsized effect on public beliefs

There’s an obvious remedy to a company that is too big to audit: break it up into smaller pieces

4. Bypassing our rational faculties

This is the good stuff: using machine learning, “dark patterns,” engagement hacking, and other techniques to get us to do things that run counter to our better judgment. This is mind control. Some of these techniques have proven devastatingly effective*

Games are extraordinarily good at this. “Free to play” games manipulate us through many techniques

Companies have risen and fallen on these techniques, and the “fallen” part is worth paying attention to. In general, living things adapt to stimulus: Something that is very compelling or noteworthy when you first encounter it fades with repetition until you stop noticing it altogether

That’s why behavioral conditioning uses “intermittent reinforcement schedules.”

Intermittent reinforcement is a powerful behavioral tool, but it also represents a collective action problem for surveillance capitalism

quickly copied across the whole sector

Who can forget the Great Zynga Epidemic, when all of our friends were caught in FarmVille ’s endless, mindless dopamine loops? But every new attention-commanding technique is jumped on by the whole industry and used so indiscriminately that antibiotic resistance sets in

by 2013, two years after Zynga’s peak, its user base had halved

surveillance capitalism’s margins on behavioral modification suck.

And new powerful attention weapons aren’t easy to find, as is evidenced by the long years since the last time Zynga had a hit. Despite the hundreds of millions of dollars that Zynga has to spend on developing new tools to blast through our adaptation, it has never managed to repeat the lucky accident that let it snag so much of our attention for a brief moment in 2009.

The vulnerability of small segments of the population to dramatic, efficient corporate manipulation is a real concern that’s worthy of our attention and energy. But it’s not an existential threat to society

If data is the new oil, then surveillance capitalism’s engine has a leak

This adaptation problem offers an explanation for one of surveillance capitalism’s most alarming traits: its relentless hunger for data and its endless expansion of data-gathering capabilities through the spread of sensors, online surveillance, and acquisition of data streams from third parties.

What if it’s all a Red Queen’s race where they have to run ever faster — collect ever-more data — just to stay in the same spot?

Of course, all of Big Tech’s persuasion techniques work in concert with one another, and collecting data is useful beyond mere behavioral trickery

profiling and targeting

Each phase of this process benefits from surveillance: The more data they have, the more precisely they can profile you and target you with specific messages.

There is one way in which targeted advertising uniquely benefits those advocating for socially unacceptable causes: It is invisible

Online ads are placed by algorithms that broker between a diverse ecosystem of self-serve ad platforms that anyone can buy an ad through

These layers of indirection between advertisers and publishers serve as moral buffers: Today’s moral consensus is largely that publishers shouldn’t be held responsible for the ads that appear on their pages because they’re not actively choosing to put those ads there. Because of this, Nazis are able to overcome significant barriers to organizing their movement

More importantly, if you can dominate the information space while also gathering data, then you make other deceptive tactics stronger because it’s harder to break out of the web of deceit you’re spinning. Domination — that is, ultimately becoming a monopoly — and not the data itself is the supercharger that makes every tactic worth pursuing because monopolistic domination deprives your target of an escape route.

Surveillance capitalists are like stage mentalists who claim that their extraordinary insights into human behavior let them guess the word that you wrote down and folded up in your pocket but who really use shills, hidden cameras, sleight of hand, and brute-force memorization to amaze you.

Pick-up artists are proof that people can believe they have developed a system of mind control even when it doesn’t work . Pick-up artists simply exploit the fact that one-in-a-million chances can come through for you if you make a million attempts

Department store pioneer John Wanamaker is said to have lamented, “Half the money I spend on advertising is wasted; the trouble is I don’t know which half.”

What is Facebook?

Facebook alone among the Western tech giants has built a business based on locking in its users and spying on them all the time

the apps — games, fart machines, business review services, apps for keeping abreast of your kid’s schooling — you use will send information about your activities to Facebook even if you don’t have a Facebook account and even if you don’t download or use Facebook apps. On top of all that, Facebook buys data from third-party brokers on shopping habits, physical location, use of “loyalty” programs, financial transactions, etc., and cross-references that with the dossiers it develops on activity on Facebook and with apps and the public web.

To understand what role Facebook plays in the formulation and mobilization of antisocial movements, we need to understand the dual nature of Facebook.

Because it has a lot of users and a lot of data about those users, Facebook is a very efficient tool for locating people with hard-to-find traits

Facebook also makes it much easier to find people who hold the same rare political beliefs as you

All of this presents a dilemma for Facebook: Targeting makes the company’s ads more effective than traditional ads, but it also lets advertisers see just how effective their ads are.

To make things worse, many Facebook groups spark precious little discussion

With nothing but “organic” discussion, Facebook would not generate enough traffic to sell enough ads to make the money

So Facebook has to gin up traffic by sidetracking its own forums

Facebook is optimized for engagement, not happiness, and it turns out that automated systems are pretty good at figuring out things that people will get angry about

Monopoly and the right to the future tense

Zuboff and her cohort are particularly alarmed at the extent to which surveillance allows corporations to influence our decisions, taking away something she poetically calls “the right to the future tense” — that is, the right to decide for yourself what you will do in the future

Zuboff puts enormous and undue weight on the persuasive power of surveillance-based influence techniques

By contrast, Zuboff is rather sanguine about 40 years of lax antitrust practice that has allowed a handful of companies to dominate the internet

if we are to be alarmed that we might lose the right to choose for ourselves what our future will hold, then monopoly’s nonspeculative, concrete, here-and-now harms should be front and center in our debate over tech policy.

Start with “digital rights management.” In 1998, Bill Clinton signed the Digital Millennium Copyright Act (DMCA) into law

Section 1201, the “anti-circumvention” rule. This is a blanket ban on tampering with systems that restrict access to copyrighted works*

The activities that the DMCA’s Section 1201 sets out to ban are not copyright infringements; rather, they are legal activities that frustrate manufacturers’ commercial plans

For example, Section 1201’s first major application was on DVD players as a means of enforcing the region coding built into those devices

However, watching a lawfully produced disc in a country other than the one where you purchased it is not copyright infringement — it’s the opposite

The fact that a movie studio wants to charge Indians less than Americans or release in Australia later than it releases in the U.K. has no bearing on copyright law

companies can control their customers’ behavior after they take home their purchases by designing products so that all unpermitted uses require modifications that fall afoul of Section 1201

This manifests in many ways: from a new generation of inkjet printers that use countermeasures to prevent third-party ink that cannot be bypassed without legal risks to similar systems in tractors that prevent third-party technicians from swapping in the manufacturer’s own parts that are not recognized by the tractor’s control system until it is supplied with a manufacturer’s unlock code.

Closer to home, Apple’s iPhones use these measures to prevent both third-party service and third-party software installation

The App Store’s commercial terms guarantee Apple a share of all revenues generated by the apps sold there

Crucially, Apple’s use of copyright locks gives it the power to make editorial decisions about which apps you may and may not install on your own device. Apple has used this power to reject dictionaries for containing obscene words; to limit political speech , especially from apps that make sensitive political commentary such as an app that notifies you every time a U.S. drone kills someone somewhere in the world; and to object to a game that commented on the Israel-Palestine conflict.

In China, the government ordered Apple to prohibit the sale of privacy tools like VPNs with the exception of VPNs that had deliberately introduced flaws designed to let the Chinese state eavesdrop on users. Because Apple uses technological countermeasures — with legal backstops — to block customers from installing unauthorized apps, Chinese iPhone owners cannot readily (or legally) acquire VPNs that would protect them from Chinese state snooping.

Search order and the right to the future tense

Many of the company’s key divisions, such as the advertising technology of DoubleClick, violate the historical antitrust principle of structural separation, which forbade firms from owning subsidiaries that competed with their customers

If we’re worried about giant companies subverting markets by stripping consumers of their ability to make free choices, then vigorous antitrust enforcement seems like an excellent remedy

This goes for many other companies. Amazon, a classic surveillance capitalist

Not every monopolist is a surveillance capitalist, but that doesn’t mean they’re not able to shape consumer choices in wide-ranging ways. Zuboff lauds Apple for its App Store and iTunes Store

But Apple is the only retailer allowed to sell on its platforms

Because of its use of copyright locks, Apple’s mobile customers are not legally allowed to switch to a rival retailer for its apps if they want to do so on an iPhone

Monopolists can afford sleeping pills for watchdogs