Reading Weapons of Math Destruction, part 4

With this post we continue our reading of Cathy O’Neil’s Weapons of Math Destruction.  This week’s chapters address the role of data analytics in the world of work.

(If you’d like to catch up with the reading schedule, click here.  All posts for this reading, including the schedule one, are grouped here.)

Here I’ll summarize this week’s chapters, then offer some discussion questions.

But first, some book club business.  There are good comments on last week’s post.  Jason Green has caught up with two thoughtful blog posts on earlier chapters (1, 2).  Mike Richi blogs his answers to some of my questions.

Onward!

Chapter 6, “Ineligible to Serve: Getting a Job” 

Two stories anchor this chapter, one concerning a man prevented from being hired because of answers to a standard questionnaire based on big data, the other about a late 20th-century British hospital’s experience with using early digital data to make hiring more efficient.  The key point is that companies all too often rely on (at best) flawed software to make hiring choices.

To begin with, these WMDs use imprecise metrics, proxies standing in for actual hiring data, such as personality tests (108, 119).  Those proxies not only fail to capture the problem, but also come close to being illegal.  Companies also don’t implement the software in a way that provides useful feedback (110-111).  On top of that, they also work at enormous scale (112).  It’s another case where an organization chooses efficiency over fairness (116).

  In practice, people with money and/or knowledge of the system can game it, “one more example in which the wealthy and informed get the edge, and the poor are more likely to lose out.” (114)

The chapter includes a pleasantly surprising positive story, wherein Xerox started a big data/data analytics project, and found it discriminated against poor people.  So they got rid of the problematic data (119).

Chapter 7, “Sweating Bullets: On the Job”

Here Weapons of Math Destruction turns to the use of data analytics upon employees.  O’Neil begins with software creating inhumane work schedules, designed to minimize compensation in favor of maximized sales.  This isn’t wholly a new development, drawing on operations research and Taylorized management (the latter oddly unmentioned); what’s new is massively more powerful data gathering and analysis (126-8).

The chapter shows how such labor management software creates “a poisonous feedback loop” which makes it harder for poor people to get ahead on the job or elsewhere.  It has negative impacts on workers’ families as well (129-130).

O’Neil then describes new software to manage white collar workers.  At least one tool uses practices drawn from knowledge management to rate staff performance, and did so in some WMD fashions: with poor data and missing feedback loops.  This subtype of data analytics could well grow into “an industrial standard… and then we’ll all be in trouble.” (134)

As a hint of that troubled future two education stories conclude the chapter, one concerning the famous Nation at Risk report (1983).  O’Neil describes its appearance and large influence, then its less well known partial debunking. (134-7)  Next she describes the bad contruction of data modeling to assess K-12 teacher quality, which ultimately “measured nothing” (emphasis in original) (137).

NB: the chapter begins with a neologism, “clopening“.  That’s when a worker is scheduled to close out their workplace one night, then open it the next.  I used to do this back in the 1980s, when working at a bookstore.  It’s not easy.

Questions

• Chapter 7 concludes by mentioning a boycott against on particular WMD implementation, and that it succeeded in pushing back its deployment (140).  Do you think boycotts could be a useful tool for resisting bad data analytics?
• Tom Haymes has been thinking about the intersection of storytelling and big data.  What’s the narrative these WMD providers are proferring?
• O’Neil describes how businesses often exploit WMDs to maximize their bottom line.  Is “neoliberalism” another word for what she’s describing?
  buy zydena online buy zydena no prescription generic

Next up: for November 13, chapter 8, “Collateral Damage: Landing Credit” and chapter 9, “No Safe Zone: Getting Insurance”.

Liked it? Take a second to support Bryan Alexander on Patreon!