The best books on information theory

James V. Stone Author Of Information Theory: A Tutorial Introduction
By James V. Stone

Who am I?

My primary interest is in brain function. Because the principal job of the brain is to process information, it is necessary to define exactly what information is. For that, there is no substitute for Claude Shannon’s theory of information. This theory is not only quite remarkable in its own right, but it is essential for telecoms, computers, machine learning (and understanding brain function). I have written ten "tutorial introduction" books, on topics which vary from quantum mechanics to AI. In a parallel universe, I am still an Associate Professor at the University of Sheffield, England.

I wrote...

Information Theory: A Tutorial Introduction

By James V. Stone,

Book cover of Information Theory: A Tutorial Introduction

What is my book about?

Originally developed by Claude Shannon in the 1940s, information theory laid the foundations for the digital revolution, and is now an essential tool in telecommunications, genetics, linguistics, brain sciences, and deep space communication. In this richly illustrated book, accessible examples are used to introduce information theory in terms of everyday games like ‘20 questions’ before more advanced topics are explored. Online MatLab and Python computer programs provide hands-on experience of information theory in action, and PowerPoint slides give support for teaching.

Written in an informal style, with a comprehensive glossary and tutorial appendices, this text is an ideal primer for novices who wish to learn the essential principles and applications of information theory.

The Books I Picked & Why

Shepherd is readers supported. When you buy through links on our website, we may earn an affiliate commission. This is how we fund this project for readers and authors (learn more).

Introduction to Information Theory: Symbols, Signals and Noise

By John R. Pierce,

Book cover of Introduction to Information Theory: Symbols, Signals and Noise

Why this book?

Pierce was a contemporary of Claude Shannon (inventor of information theory), so he learned information theory shortly after it was published in 1949. Pierce writes in an informal style, but does not flinch from presenting the fundamental theorems of information theory. Some would say his style is too wordy, and the ratio of words/equations is certainly very high. Nevertheless, this book provides a solid introduction to information theory. It was originally published in 1961, so it is a little dated in terms of topics covered. However, because it was re-published by Dover in 1981, it is also fairly cheap. Overall, this is a sensible first book to read on information theory.

An Introduction to Information Theory

By Fazlollah M. Reza,

Book cover of An Introduction to Information Theory

Why this book?

This is a more comprehensive and mathematically rigorous book than Pierce’s book. For the novice, it should be read-only after first reading Pierce’s more informal text. Due to its vintage, the layout is fairly cramped, but the content is impeccable. At almost 500 pages, it covers a huge amount of material. This was my main reference book on information theory for many years, but it now sits alongside more recent texts, like MacKay’s book (see below). It is also published by Dover, so it is reasonably priced.

Elements of Information Theory

By Thomas M. Cover, Joy A. Thomas,

Book cover of Elements of Information Theory

Why this book?

This is the modern standard text on information theory. It is both comprehensive and highly technical. The layout is spacey, and the authors make good use of the occasional diagram to explain geometric aspects of information theory. One feature I really like is the set of historical notes and a summary of equations at the end of each chapter.

Information Theory, Inference and Learning Algorithms

By David JC MacKay,

Book cover of Information Theory, Inference and Learning Algorithms

Why this book?

This is considered to be a modern classic on information theory. It is a fairly readable text that roams far and wide over many topics. MacKay was extraordinarily clever, curious, and generous (which is why he made the book freely available). His wide range of interests included coding theory and artificial neural networks (machine learning), which occupy a few chapters of the book. In describing the difference between Bayesian and frequentist statistical methods, MacKay pulls no punches, and he can be heard laughing through gritted teeth as he tries to make sense of the nonsensical. This is one of those books in which the author’s many throw-away asides are as enlightening as the main text. The book’s website (below) also has a link to an excellent series of video lectures by MacKay. It is also available for free, just use the direct link below.

The Mathematical Theory of Communication

By Claude E. Shannon, Warren Weaver,

Book cover of The Mathematical Theory of Communication

Why this book?

This is really two books, strapped together. The first book by Weaver is an informal introduction to the ideas implicit in the second book by Shannon. Shannon’s book is naturally quite dated in its use of language (e.g. uncertainty is pronounced equivocation), and in its references to ‘current’ technology (e.g. PCM). Despite these caveats, it is still a surprisingly accessible book. Finally, because information theory was developed almost exclusively by Claude Shannon, reading the theory explained in his own words gives some insight into how on Earth he managed to come up with such a radical set of ideas.

Closely Related Book Lists