_Weapons of Math Destruction_, part 1

With this post we commence our reading of Cathy O’Neil’s Weapons of Math Destruction.  (If you’d like to catch up with the reading schedule, click here.)

Here I’ll summarize this week’s chapters, then offer some discussion questions.

But first, some book club business.  It’s great to see a bunch of people have expressed a desire to read along, like this nice person on Twitter and on their blog:

If you’d like further resources about this book, EconTalk has a fine interview with O’Neil. (thanks to Bob Calder). The excellent librarian (and crime novel scholar) Barbara Fister published a fine review at Inside Higher EdChris Newfield (a Future Trends Forum guest) co-authored a review article including WMD.



Here O’Neil introduces herself and the book’s major themes.  For autobiography, the author describes her childhood love of numbers, her academic career leading to a tenure-track position, a big career jump to work for a Wall Street hedge fund, working for an ecommerce startup, blogging, and joining Occupy Wall Street.

For the book’s major themes, they concern “the dark side of Big Data.” (13)  A story introduces these, one concerning Washington D.C. schoolteachers fired based on an algorithm’s findings during Michelle Rhee‘s tenure as chancellor.   O’Neil uses this as a cautionary tale about how bad data analytics – the titular Weapons of Math Destruction – can backfire and cause human suffering.

Key problems:

  • An algorithm entering a feedback loop whereby its results are trusted because the software confirms it; (7)
  • The WMD “punishes the poor” – the wealthiest people tend to receive personal attention;
  • The algorithm cannot be explored publicly, given secrecy, as “the model itself is a black box”;(8)
  • It is more difficult to push back on an algorithm than it is to be condemned by one (10).

The proliferation of WMDs are a major ethical threat to data scientists, as the latter “all too often lose sight of the folks on the receiving end of the [data analytical] transaction.” (12)

Chapter 1, “Bomb Parts: What is a Model?”

This chapter explores data models (defined as “nothing more than an abstract representation of some process”, 18; also “opinions embedded in mathematics”, 21) using three examples.  First, Moneyball (Michael Lewis, 2003), a data analytical approach which “represents a healthy case study” of applied data science (15-19).  Second, the author’s mental model for preparing meals for her family, which is workable but not scaleable (19-22).  Third, the story of predictive sentencing software, which embodies racism (23-30).  All rely on math for its objectivity.

The chapter uses these cases to reiterate the introduction’s criteria for determining a WMD’s quality:

  1. Transparency.  The Moneyball model is based on publicly accessible data, while the prison sentencing data is hard to get and the analytics closely guarded.
  2. Statistical rigor.  There has to be a big enough and relevant data set.
  3. A learning curve.  New data gets fed into the system, which adjusts itself accordingly.
  4. Damage.  How many people are hurt by the system?


What does the model of algorithms presented so far tell us about social media?

Have you had experiences with big data that O’Neil’s account illuminates?

What would it take for an education algorithm to meet all of O’Neil’s criteria for not doing damage?

What are the best ways to address the problem of “false positives”, of exceptionally bad results, of anomalies?

Liked it? Take a second to support Bryan Alexander on Patreon!
Become a patron at Patreon!
This entry was posted in readings and tagged . Bookmark the permalink.

11 Responses to _Weapons of Math Destruction_, part 1

  1. Tom Haymes says:

    I think it’s interesting that a number of the cases O’Neil described in her book ultimately fail. In my opinion this is because they disintermediate the human aspect of the equation.

    I wrote a blog entry last week on how the Houston Astros have integrated data analytics with human intermediaries (in this case Manager A.J. Hinch) and how that human element is critical in actually improving the performance of the team. I take that analogy and apply it to the role of the teacher in intermediating between students and the data they are confronted with in a class. http://www.pbk.com/#!/insight/what-houston-astros-can-teach-us-about-integrating/

    There were people warning about the instability of various markets before 2008 crash and no one listened to them. They were the “good” teachers who perceived patterns in the data that the computers ignored because they were simply executing instructions with narrow parameters. It’s possible that computers will get better at this over time but humans continue to be the champions in pattern recognition. We just need to educate them not to be intimidated by the data…

  2. VanessaVaile says:

    My own distribution is rather more makeshift (Rube Goldberg some might say) than Tom’s. Starting with Water Knife, I’ve been bundling links for FTTE Book Club readings, saving bundles as permalinks, bookmarking on Diigo and sending FTTE-reading tag rss to InoReader. Bundles include both Bryan’s blog posts, other posts about the reading and related/background reading links. I stashed the feed in a miscellaneous books and writers (overflow) folder. With a few more steps, I can create an html clip or bundle for the collection.

    That still leaves DeMillo’s Revolution in Higher Education, Putnam’s Our Kids, Brynjolfsson and McAfee’s The Second Machine Age, Solnit’s River of Shadows, and Piketty’s Capital in the 21st Century to catch up with ~ no telling when I’ll get to those though.

  3. chairthrower says:

    I would love to spend more time discussing Michelle Rhee as the embodiment of WMD thinking–while O’Neill mentions it in the introduction it’s a fairly damning commentary on the kind of education reform currently ripping through our public schools (and only our public schools, for the most part). In NJ we have the PARCC and teacher evaluations are currently tied to it, but it has no consequence for the students yet (although it will eventually be a graduation requirement). So students can literally try and tank a teacher by blowing the test. Also, I keep thinking of my wife’s former students (she teaches AP History) who come back 5 or more years later and tell her how much they now value what she did–and if you asked them at the time, or used standardized test scores, my wife may not have been rated nearly as highly.

    Fundamentally, there’s no easy way to measure teacher progress, and the best you could do is a 5-10 year longitudinal study.

  4. VanessaVaile says:

    Reblogged this on As the Adjunctiverse Turns and commented:
    Ready for some ed reading?

  5. Pingback: Book Club–“Weapons of Math Destruction” by Cathy O’Neil – Mike Richichi Dot Net

  6. Pingback: _Weapons of Math Destruction_, part 2 | Bryan Alexander

  7. Pingback: _Weapons of Math Destruction_, part 3 | Bryan Alexander

Leave a Reply

Your email address will not be published. Required fields are marked *