The best books that keep an AI researcher awake at night

I have always been fascinated by feedback loops, so even though the positive feedbacks discussed in this book have a very negative impact on society, they stir my technical curiosity.

More importantly, Ward highlights a very real problem with AI technology and its deployment: the potential for automated recommendations and other nudges to move us, as a population, toward conformity and predictability. 

Ward actually writes about 3 nested loops: 1) our evolved human tendencies, 2) the exploitation of those tendencies by human-based enterprises, such as capitalism and marketing, and 3) exploitation of those same tendencies by AI tools.

This book is very long, and somewhat redundant at times. But it’s extremely interesting…and chilling. 

Zuboff cites a wide variety of examples of how companies, Google foremost among them, gather information about us (legally or illegally) and then use it not only to predict our behavior, but to control it as well. That’s the really scary part.

The writing can be a bit too poetic at times, but Zuboff displays an incredible breadth and depth of knowledge on this subject. I’m a slow reader, so this one took me a while to get through, but it was time well spent.

Although it is now a well-known fact that many actors are fighting for our online attention, and will do just about anything to get and keep it, Wu puts it all in historical perspective by going back to the 1800’s and the beginning of print advertising. 

He then traces our relationships with ads across a century and four screens: movies, television, home computers, and finally the cell phone. For the same reason that I enjoy reading history books to try to make some sense of the world’s current political chaos, this masterpiece by Wu should ensure you that nothing about human greed has changed in any major way: the tools of exploitation just get more powerful and more addictive.

My first 3 picks put much of the blame for widespread attention capture on greedy actors engaged in a “race to the bottom of the brainstem” (Tristan Harris, former Google employee, became popular opponent of exploitative big tech). 

Although Carr does not shy from that theme, he seems to put much of the responsibility on our own shoulders: it is we who must resist many temptations of convenience in order to preserve our own cognitive strengths, such as creativity. Otherwise, we become shallow thinkers and reliant on AI even for something as characteristically human as wisdom. 

Carr’s follow-up to this book, The Glass Cage, is also very intriguing, and it offers a little more hope for a humanity that is currently bound up in a web of frightening technological dependency.

In my mind, Carr nails the real threats of AI: a total dumming-down of humanity to a level where we behave like AI and are thus incredibly predictable and malleable – putty in the hands of any public or private organization that wants to exploit us, without an inkling of our awareness.

I read this and Bostom’s SuperIntelligence around the same period. 

Whereas the latter was a bit too sci-fi for me – and Bostom’s paper-clip-generating monster has become quite the overused meme – I feel that Barrat gets into a much more realistic set of danger scenarios for AI’s that may not take over the world but can cause massive disruption, as things like trader bots already have. 

Perhaps most prescient is his discussion of the containment problem (with an expert on the topic) in light of a recent Chat system that managed to fool a human into giving it access to a robot-prohibited website. I also appreciate Barrat’s interviews of a wide range of AI experts. Although others have interviewed many such luminaries (with Martin Ford’s Architects of Intelligence being the most comprehensive), I particularly enjoy the way that Barrat weaves it all together.

This may only come in 5th on my list because I read it a long time ago. A reread might push it to the top.