The best books for data-driven enthusiasts, and believers in Twyman’s Law

Most people and organizations aspire to be data-driven, yet some of the most fascinating “discoveries” and stories are incorrect. 

The authors use the term Bullshit; I prefer Twyman’s law: any figure that looks interesting or different is usually wrong—look at very surprising results with skepticism. For instance, the adage that correlation doesn’t imply causation is well known, but correlations don’t help sell new articles, so many correlations are told as causal stories.

The book elucidates pitfalls from uncontrolled experiments, common causes, p-hacking, and selection bias, and amazing “classical” stories are debunked.

Example: If you claim your toothpaste reduces plaque “by up to” 50 percent, the only way that would be false is if the toothpaste worked too well.

This insightful book on evidence-based management is rich with great references. It is filled with provoking examples of things we heard or learned, which were debunked, making you think twice about them (Twyman’s law).

Do financial incentives work? They can, but in limited scenarios where individuals could be measured and minimum quality levels kept high; they may, however, backfire and undermine teamwork and attract the wrong people. When hiring, can the best talent be identified? Even in sports, some of the best players aren’t identified early in their careers. 

Rembrandt is considered a great painter; Mozart, one of the greatest composers, but they were largely unrecognized and unrewarded during their lifetimes.  Does supervisor feedback help? In some scenarios, only the supervisors think so.

A theme in the book is that “Most people, when directly confronted by evidence that they are wrong, do not change their point of view or course of action but justify it even more tenaciously.”  

In software, we have conclusive data from tens of thousands of online controlled experiments (A/B tests) that most (~80%) ideas and features proposed fail to improve the metrics they were designed to improve. 

The book walks us through great examples on the cognitive dissonance that occurs when our perceived expertise clashes with the sobering reality of failure—humbling. The key, of course, is to think of these experiments as learning opportunities: you win some, you learn some.

Nassim Taleb calls this book “one of the most important management books of all time, and an antidote to these bestselling books by gurus presenting false patterns and naïve arguments.” 

Rosenzweig shows that although best-selling management books are inspirational, they are written based on correlational data and lack predictive power. For instance: “Does employee satisfaction lead to high performance? The evidence suggests it’s mainly the other way around—company success has a stronger impact on employee satisfaction.” 

The problem with only studying the “best,” as management classics like In Search of Excellence have done, is that there is no experimental control group. If rich people fly private jets, starting to fly similarly is more likely to lead to bankruptcy than wealth. 

This great management book was written by an engineer who clearly explains the rationale for his recommendations.

Andy Grove, former chairman and CEO of Intel, is credited with driving the growth phase of Silicon Valley, was named Time’s Man of the Year, and is credited as the “Father of OKRs.” One example that I love: a new hire does poor work. His manager says: “He has to make his own mistakes…that’s how he learns!”

Grove writes: “absolutely wrong…the tuition is paid by his customers…[instead of by the manager].” Another gem: “Review rough drafts [you delegated]; don't wait until your subordinates have spent time polishing them into final form before you find out that you have a basic problem with the contents.”