With this post we continue our reading of Cathy O’Neil’s Weapons of Math Destruction. Now the topic shifts to the impact of data analytics on personal finance.
(If you’d like to catch up on the reading schedule, click here. All posts for this reading, including the schedule one, are grouped here. More info about our online book club is here.)
Here I’ll summarize this week’s chapters, then offer some discussion questions.
But first, some book club contributions since the last post. The New Inquiry released a big data project that inverts one key piece of a WMD, by focusing on the wealthy instead of the poor by mapping the likelihood of white collar crime (white paper) (thanks to Jason Green). Slate has a new article about the risks of predictive policing (thanks to Jason again). Jason also blogged about part 3 of our reading.
On Twitter Duncan Stewart issued this overview of the book:
A good argument for algorithmic caution.
— Duncan Stuart (@duncan_stuart) November 7, 2017
Sherri Spelic shared a feeling of gloom at this point in the reading:
Also reading this right now. Hard not to feel utterly overwhelmed. Necessary read, though. https://t.co/IC7SugnZwN
— Sherri Spelic (@edifiedlistener) November 6, 2017
Good thoughts. Once again, I’m delighted to hear from so many people as we read together.
Now, onward to this week’s reading…
Chapter 8, “Collateral Damage: Landing Credit”
This chapter explores the ways companies can (mis)assess our financial status. Key to this is the idea of “e-scores”, or methods for collecting and interpreting creditworthiness that do not include actual credit scores (143, 144). One problem with them is their reliance on “a veritable blizzard of proxies” instead of more relevant data, especially based on classes of people, rather than on individuals (145-6). Once again this functions as a negative feedback loop as bad e-scores make it harder for poor people and people of color to escape poverty (148-9, 158). E-score data is also largely unregulated (151).
Also in this chapter is an anti-WMD, FICO scores, as O’Neil views them as transparent, effectively regulated, and working with “a clear feedback loop” (142). Another positive feature is the role of individuals reaching through data to glimpse the lives of others, and act to complement a WMD (160, 163,165).
Keep an eye out for e-scores; they will recur through the rest of this book.
Chapter 9, “No Safe Zone: Getting Insurance”
This chapter picks up on the previous ones to further pursue problems of data analytics on personal finance, with a focus here on insurance. That’s a business with a longstanding focus on data, and O’Neil begins with an early example of it going wrong by establishing redlining. More recently, insurance companies can err by classing customers incorrectly.
One way they err is when car insurance businesses rely less on how well customers driver, and much more on their financial data, according to a recent Consumer Reports investigation. Yes, e-scores return, and have a powerful, often invisible, impact on what we pay for policies (164-5). Once again, companies deploy such data analytics to extract greater profit. Their tools fit O’Neil’s WMD model, being anti-transparent, scaled, and offering a bad feedback loop. Once more, they are tilted against poor people, making it harder for them to work their way up the ladder (169, 171).
O’Neil finds the wellness movement to partake of this bad data science. Despite good reasons for using it as a way to encourage people to lead healthier lives, it leads to intrusive surveillance. The author fears that firms could use wellness data to shape employment hiring decisions (175), but does hold back from condemning wellness programs as full WMDs, finding them actually pretty transparent (178).
Body mass index (BMI) comes in for criticism as well (176-7).
Questions
- What are good examples of positive uses of medical data?
- Do you see e-scores or similar data analytics in your world?
- What prevents these tools from being transparent?
Next up: for November 20: chapter 10, “The Targeted Citizen: Civic Life”, the conclusion, and the afterward (apparently new for the paperback edition).
Reblogged this on As the Adjunctiverse Turns and commented:
ICYMI…never too late to start reading. Algorithms permeate higher ed and everywhere else.
Cathy O’Neil has a column in today’s NYTimes calling for more academic research into WMDs including formulating ethical standards.
https://www.nytimes.com/2017/11/14/opinion/academia-tech-algorithms.html?_r=0
Good catch! Adding to next week’s post.
Hm. There are academics doing this work.