Custodians

I’m thrilled to say that my new book, Custodians of the Internet, is now available for purchase from Yale University Press, and your favorite book retailer. Those of you who know me know that I’ve been working on this book for a long time, and have cared about the issues it addresses for a while now. So I’m particularly excited that it is now no longer mine, but yours if you want it. I hope it’ll be of some value to those of you who are interested in interrogating and transforming the information landscape in which we find ourselves.

By way of introduction, I thought I would explain the book’s title, particularly my choice of the word “custodians.” This title came unnervingly late in the writing process, and after many, many conversations with my extremely patient friend and colleague Dylan Mulvin. “Custodians of the Internet” captured, better than many, many alternatives, the aspirations of social media platforms, the position they find themselves in, and my notion for how they should move forward.

moderators are the web’s “custodians,” quietly cleaning up the mess: The book begins with a quote from one of my earliest interviews, with a member of YouTube’s content policy team. As they put it, “In the ideal world, I think that our job in terms of a moderating function would be really to be able to just turn the lights on and off and sweep the floors . . . but there are always the edge cases, that are gray.” The image invoked is a custodian in the janitorial sense, doing the simple, mundane, and uncontroversial task of sweeping the floors. In this turn of phrase, content moderation was offered up as simple maintenance. It is not imagined to be difficult to know what needs scrubbing, and the process is routine. As with content moderation, there is labor involved, but largely invisible, just as actual janitorial staff are often instructed to “disappear,” working at night or with as little intrusion as possible. yet even then, years before Gamergate or ISIS beheadings or white nationalists or fake news, it was clear that moderation is not so simple.

platforms have taken “custody” of the Internet: Content moderation at the major platforms matters because those platforms have achieved such prominence in the intervening years.As I was writing the book, one news item in 2015 stuck with me: in a survey on people’s new media use, more people said that they used Facebook than said they used the Internet. Facebook, which by then had become one of the most popular online destinations in the world and had expanded to the mobile environment, did not “seem” like the Internet anymore. Rather than being part of the Internet, it had somehow surpassed it. This was not true, of course; Facebook and the other major platforms had in fact woven themselves deeper into the Internet, by distributing cookies, offering secure login mechanisms for other sites and platforms, expanding advertising networks, collecting reams of user data from third-party sites, and even exploring Internet architecture projects. In both the perception of users and in material ways, Facebook and the major social media platforms have taken “custody” of the Internet. This should change our calculus as to whether platform moderation is or is not “censorship,” and the responsibilities of platforms bear when they decide what to remove and who to exclude.

platforms should be better “custodians,” committed guardians of our struggles over value: In the book, I propose that these responsibilities have expanded. Users have become more acutely aware, of both the harms they encounter on these platforms, and the costs of being wronged by content moderation decisions. What’s more, social media platforms have become the place where a variety of speech coalitions do battle: activists, trolls, white nationalists, advertisers, abusers, even the President. And the implications of content moderation have expanded, from individual concerns to public ones. If a platform fails to moderate, everyone can be affected, even those who aren’t party to the circulation of the offensive, the fraudulent, or the hateful — even those who aren’t on social media at all.

What would it mean for platforms to play host not just to our content, but to our best intentions? The major platforms I discuss here have, for years, tried to position themselves as open and impartial conduits of information, defenders of their user’s right to speak, and legally shielded from any obligations for how they police their sites. As most platform managers see it, moderation should be theirs to do, conducted on their own terms, on our behalf, and behind the scenes. But that arrangement is crumbling, as critics begin to examine the responsibilities social media platforms have to the public they serve.

In the book, I propose that platforms become “custodians” of the public discourse they facilitate — not in the janitorial sense, but something more akin to legal guardianship. The custodian, given charge over a property, a company, a person, or a valuable resource, does not take it for their own or impose their will over it; they accept responsibility for ensuring that it is governed properly. This is akin to Jack Balkin’s suggestion that platforms act as “information fiduciaries,” with a greater obligation to protect our data. But I don’t just mean that platforms should be custodians of our content; platforms should be custodians of the deliberative process we all must engage in, that makes us a functioning public. Users need to be more accountable for making the hard decisions about what does and does not belong; platforms could facilitate that deliberation, and then faithfully enact the conclusions users reach. Safeguarding public discourse requires ensuring that it is governed by those to whom it belongs, making sure it survives, that its value is sustained in a fair and equitable way. Platforms could be not the police of our reckless chatter, but the trusted agents of our own interest in forming more democratic publics.

If you end up reading the book, you have my gratitude. And I’m eager to hear from anyone who has thoughts, comments, praise, criticism, and suggestions. You can find me on Twitter at @TarletonG.

Can objects be evil? A review of “Addiction by Design”

Bliss by Sean O'Brien
Bliss by Sean O’Brien

Schüll, Natasha Dow. (2012) Addiction by Design: Machine Gambling in Las Vegas. Princeton: Princeton University Press.

Addiction by Design is a nonfiction page-turner. A richly detailed account of the particulars of video gaming addiction, worth reading for the excellence of the ethnographic narrative alone, it is also an empirically rigorous examination of users, designers, and objects that deepens practical and philosophical questions about the capacities of players interacting with machines designed to entrance them. Many books that make worthy contributions to the theoretical literature of a particular field are slogs to read. Addiction by Design is as compelling as a horror story—a sad, smart horror story that calls off the Luddite witch hunt (Down with the machines!) in favor of an approach that examines the role of gaming designers within existing social systems of gender and class disparity.

The most popular gaming machines serve up video slots and video poker. They run on paycards because inserting cash and coinage slows down the rate of play, compromising the experience. By the mid-1990s in Las Vegas, Schüll reports, the vast majority of people at Gamblers Anonymous meetings were addicted to machines—not the table games, ponies, or lotteries previously associated with problem gambling. In 2003 it was estimated that 85 percent of industry profits nationally came from video gaming. For the people (mostly women) who become addicts, the draw of the machines has little to do with the possibility of winning big. Problem gamblers are attracted to the machines because they offer portals to an appealing parallel universe in which they can disconnect from the anxieties and pressures of everyday life. One of Schüll’s interviewees, Mollie, explains, “It’s like being in the eye of a storm, is how I’d describe it. Your vision is clear on the machine in front of you but the whole world is spinning around you, and you can’t really hear anything. You aren’t really there—you’re with the machine and that’s all you’re with.”

woman playing video slots
Woman playing video slots, 2010, Copyright Kate Krueger

Mollie’s experience is typical in at least two ways. First, she has a traumatic past that predisposes her to addictive behaviors. Second, she repeatedly spends all that she has in binges. But before we blame Mollie and other victims and then expound the benefits of 12-step programs with earnest optimism, Schüll asks readers to consider the insidious dependencies that arise between machine designers, casino owners, and gamblers, especially “problem gamblers,” whose struggle to control personal spending generates 30 to 60 percent of casino revenue. Schüll’s Addiction systematically builds on her basic argument that, “just as certain individuals are more vulnerable to addiction than others, it is also the case that some objects, by virtue of their unique pharmacologic or structural characteristics, are more likely than others to trigger or accelerate an addiction.”

Schüll describes the progression of changes the industry has introduced in search of higher profits. For a while, ergonomics was economics. Then high-priced animators were hired to design pleasing sounds and animations to reward winners. But some players were annoyed that the animations were too slow, so the animations were dropped. Play sped up. Faster play was great for increasing dopamine delivery to the brain. It also tended to speed players toward the end of their credits, which lowered their loyalty to particular machines and the casinos that housed them. Chip-driven gaming allowed designers to respond to this problem by tweaking the programs so that frequent small wins (often less than the cost of playing a single hand) kept dopamine surging while players’ cash trickled steadily into casino coffers. One player in a gambling support group compared video machines to crack cocaine, a comparison frequently repeated by researchers and psychologists.[1] By some accounts, the recidivism rate is now higher for gambling than for any other addiction.

The demons here are not the machines, though they are manifest in the machines. The demons are not the people who design the machines nor the people who build palaces in which the machines are arrayed in blinking, burbling gardens of vertiginous electronica. The demons are not located in the players’ genes or childhoods. The demons are not the state regulators who now embrace video gaming after corralling it on American Indian reservations for decades. There is no single devil here, and no particular exorcism can right the wrong, but there is something devilish about the way designers’ intentions and users’ neurology meet up to make video gaming so devastating for some and so profitable for others.

[1] Mary Sojourner, She Bets Her Life: A True Story of Gambling Addiction (New York: Seal Books, 2010).

This review is cross-posted at publicbooks.org, a new book review and visual essay website affiliated with Public Culture, a peer-reviewed academic journal.