With this post we continue our reading of Cathy O’Neil’s Weapons of Math Destruction. (If you’d like to catch up with the reading schedule, click here. All posts for this reading, including the schedule one, are grouped here.)
Here I’ll summarize this week’s chapters, then offer some discussion questions.
But first, some book club business. Last week’s reading elicited some fine comments. Elsewhere Mike Richichi continued his bookblogging with reflections on last week’s reading. On Twitter Mark Corbett Wilson offered this bracing thought:
Elsewhere on the web, this Motherboard article explores how bias works in one of Google’s APIs.
Chapter 4, “Propaganda Machine: Online Advertising”
While the chapter title invokes the general topic of online advertising, this section really focuses on one instance: its use by for-profit colleges and universities. This continues last week’s theme of data in higher education. As Tressie Cottom also argues, O’Neil finds for-profits aggressively targeting poor people. They promote “ads that pinpoint people in great need and sell them false or overpriced promises. The find inequality and feast on it.” (70) And they work with Cottom’s “gospel of education”, that widespread belief in the fruitful connection between education and career progress (81).
In terms of online advertising in general, O’Neil points out that Facebook and Google ads are interesting WMDs, because they do have some unusual features, one being learning from campaigns. They energetically learn from new data, and now “sift through data on their own… [w]ith machine learning.” (75) . They also act at very large scale, which further strengthens their ability to learn.
The end of chapter 4 touches on one other industry using predatory advertising: payday loan outfits.
Chapter 5, “Civilian Casualties: Justice in the Age of Big Data”
Here the book shifts ground from education to criminal justice, identifying a new group of bad algorithms. We lead off with Predpol, software designed to help police determine the most crimogenic areas of a city. This causes problems when applied to nuisance crimes through a broken windows policing approach, as that disproportionately targets poor neighborhoods. A feedback loop then results when analysis gathers more data from those regions, revealing more crime. “The result is that we criminalize poverty, believing all the whole that our tools are not only scientific but fair.” (91) (O’Neil offers a fine, acidic thought experiment whereby police give rich white collar areas the same treatment, 89-90).
O’Neil derives several general principles about WMDs from the policing experience, the first being that big data and data analytics involves “a choice between fairness and efficiency” (94). Efficiency is easier for software to handle. A second concept is that WMDs tend to be applied unequally. As with the previous chapter, many enterprises aim their data instruments at the poor. One more point flows from this: that we tend to gather data unevenly, ruling out certain categories because they are uncomfortable. There is, in short, always a question of which data is obtained, and which is excluded.
The chapter concludes on a recommendation for community policing, for “attempting to build relationships in the neighborhood” rather than subjecting an area to data surveillance. (103)
- Are there advertising campaigns using big data that avoid these problems?
- If non-profit higher education competition heats up, will those campuses turn to these sorts of big data campaigns?
- Mark Wilson asks a powerful question. If all data massively collected in America is biased, how should we proceed?
Next up: for November 6: chapter 6, “Ineligible to Serve: Getting a Job” and chapter 7, “Sweating Bullets: On the Job”.