Here are 100 books that The Mathematical Theory of Communication fans have personally recommended if you like
The Mathematical Theory of Communication.
Shepherd is a community of 12,000+ authors and super readers sharing their favorite books with the world.
My primary interest is in brain function. Because the principal job of
the brain is to process information, it is necessary to define exactly
what information is. For that, there is no substitute for Claude
Shannon’s theory of information. This theory is not only quite
remarkable in its own right, but it is essential for telecoms,
computers, machine learning (and understanding brain function).
I have written ten "tutorial introduction" books, on topics which vary
from quantum mechanics to AI.
In a parallel universe, I am still an Associate Professor at the
University of Sheffield, England.
Pierce was a contemporary of Claude Shannon (inventor of information theory), so he learned information theory shortly after it was published in 1949. Pierce writes in an informal style, but does not flinch from presenting the fundamental theorems of information theory. Some would say his style is too wordy, and the ratio of words/equations is certainly very high. Nevertheless, this book provides a solid introduction to information theory. It was originally published in 1961, so it is a little dated in terms of topics covered. However, because it was re-published by Dover in 1981, it is also fairly cheap. Overall, this is a sensible first book to read on information theory.
"Uncommonly good...the most satisfying discussion to be found." — Scientific American. Behind the familiar surfaces of the telephone, radio, and television lies a sophisticated and intriguing body of knowledge known as information theory. This is the theory that has permitted the rapid development of all sorts of communication, from color television to the clear transmission of photographs from the vicinity of Jupiter. Even more revolutionary progress is expected in the future. To give a solid introduction to this burgeoning field, J. R. Pierce has revised his well-received 1961 study of information theory for a second edition. Beginning with the origins…
My primary interest is in brain function. Because the principal job of
the brain is to process information, it is necessary to define exactly
what information is. For that, there is no substitute for Claude
Shannon’s theory of information. This theory is not only quite
remarkable in its own right, but it is essential for telecoms,
computers, machine learning (and understanding brain function).
I have written ten "tutorial introduction" books, on topics which vary
from quantum mechanics to AI.
In a parallel universe, I am still an Associate Professor at the
University of Sheffield, England.
This is the modern standard text on information theory. It is both comprehensive and highly technical. The layout is spacey, and the authors make good use of the occasional diagram to explain geometric aspects of information theory. One feature I really like is the set of historical notes and a summary of equations at the end of each chapter.
The latest edition of this classic is updated with new problem sets and material The Second Edition of this fundamental textbook maintains the book's tradition of clear, thought-provoking instruction. Readers are provided once again with an instructive mix of mathematics, physics, statistics, and information theory. All the essential topics in information theory are covered in detail, including entropy, data compression, channel capacity, rate distortion, network information theory, and hypothesis testing. The authors provide readers with a solid understanding of the underlying theory and applications. Problem sets and a telegraphic summary at the end of each chapter further assist readers. The…
I started my career in neuroscience. I wanted to understand brains. That is still proving difficult, and somewhere along the way, I realized my real motivation was to build things, and I wound up working in AI. I love the elegance of mathematical models of the world. Even the simplest machine learning model has complex implications, and exploring them is a joy.
The best parts of this book really represent a gold standard in pedagogical clarity.
Although it’s now twenty years old, there is still much to learn from this rather unconventional book that covers the boundary between machine learning, information theory, and Bayesian methods. There are also odd tangents and curiosities, some of which work better than others but are never dull.
Just writing this review makes me want to go back to it and squeeze more out of it.
Information theory and inference, taught together in this exciting textbook, lie at the heart of many important areas of modern technology - communication, signal processing, data mining, machine learning, pattern recognition, computational neuroscience, bioinformatics and cryptography. The book introduces theory in tandem with applications. Information theory is taught alongside practical communication systems such as arithmetic coding for data compression and sparse-graph codes for error-correction. Inference techniques, including message-passing algorithms, Monte Carlo methods and variational approximations, are developed alongside applications to clustering, convolutional codes, independent component analysis, and neural networks. Uniquely, the book covers state-of-the-art error-correcting codes, including low-density-parity-check codes, turbo…
My primary interest is in brain function. Because the principal job of
the brain is to process information, it is necessary to define exactly
what information is. For that, there is no substitute for Claude
Shannon’s theory of information. This theory is not only quite
remarkable in its own right, but it is essential for telecoms,
computers, machine learning (and understanding brain function).
I have written ten "tutorial introduction" books, on topics which vary
from quantum mechanics to AI.
In a parallel universe, I am still an Associate Professor at the
University of Sheffield, England.
This is a more comprehensive and mathematically rigorous book than Pierce’s book. For the novice, it should be read-only after first reading Pierce’s more informal text. Due to its vintage, the layout is fairly cramped, but the content is impeccable. At almost 500 pages, it covers a huge amount of material. This was my main reference book on information theory for many years, but it now sits alongside more recent texts, like MacKay’s book (see below). It is also published by Dover, so it is reasonably priced.
Written for an engineering audience, this book has a threefold purpose: (1) to present elements of modern probability theory — discrete, continuous, and stochastic; (2) to present elements of information theory with emphasis on its basic roots in probability theory; and (3) to present elements of coding theory. The emphasis throughout the book is on such basic concepts as sets, the probability measure associated with sets, sample space, random variables, information measure, and capacity. These concepts proceed from set theory to probability theory and then to information and coding theories. No formal prerequisites are required other than the usual undergraduate…
I am a financial data scientist. I think it is important that data scientists are highly specialized if they want to be effective in their careers. I run a business called Conlan Scientific out of Charlotte, NC where me and my team of financial data scientists tackle complicated machine learning problems for our clients. Quant trading is a gladiator’s arena of financial data science. Anyone can try it, but few succeed at it. I am sharing my top five list of math books that are essential to success in this field. I hope you enjoy.
This book might as well be called Introduction to machine learning, and it is probably one of the only books truly deserving of the title. Did you know neural networks have been used for decades to scan checks at the bank? They are called Boltzman Machine. Have you ever heard of how decision trees were used in old-school data mining? You could only get them from proprietary software packages from the early 2000s.
In quant trading, you will constantly face compute power constraints, so it is invaluable to understand the mathematical foundations of the most old-school machine learning methods out there. Researchers 20 years ago used to do a lot of impressive work with a lot less computing power.
This book describes the important ideas in a variety of fields such as medicine, biology, finance, and marketing in a common conceptual framework. While the approach is statistical, the emphasis is on concepts rather than mathematics. Many examples are given, with a liberal use of colour graphics. It is a valuable resource for statisticians and anyone interested in data mining in science or industry. The book's coverage is broad, from supervised learning (prediction) to unsupervised learning. The many topics include neural networks, support vector machines, classification trees and boosting---the first comprehensive treatment of this topic in any book.
I am a financial data scientist. I think it is important that data scientists are highly specialized if they want to be effective in their careers. I run a business called Conlan Scientific out of Charlotte, NC where me and my team of financial data scientists tackle complicated machine learning problems for our clients. Quant trading is a gladiator’s arena of financial data science. Anyone can try it, but few succeed at it. I am sharing my top five list of math books that are essential to success in this field. I hope you enjoy.
One of my favorite professors, Gretchen Martinet, used this to teach a course called “Mathematical Statistics” when I was at the University of Virginia. It is an extremely profound course full of dense but fundamental mathematical proofs in classical statistics.
You will learn why the formula for the normal distribution is the way it is, why the sum of squares appears everywhere in statistics, and how to fit a linear regression by hand. In the same way calculus elevates our understanding of rates of changes, the book elevates your understanding of samples, averages, and distributions. Quant trading requires an intuitive sense of how data, models, and aggregates work, making this content essential for your success.
Modern Mathematical Statistics with Applications, Second Edition strikes a balance between mathematical foundations and statistical practice. In keeping with the recommendation that every math student should study statistics and probability with an emphasis on data analysis, accomplished authors Jay Devore and Kenneth Berk make statistical concepts and methods clear and relevant through careful explanations and a broad range of applications involving real data.
The main focus of the book is on presenting and illustrating methods of inferential statistics that are useful in research. It begins with a chapter on descriptive statistics that immediately exposes the reader to real data. The…
Who Will Take Care of Me When I'm Old?
By
Joy Loverde,
Everything you need to know to plan for your own safe, financially secure, healthy, and happy old age.
For those who have no support system in place, the thought of aging without help can be a frightening, isolating prospect. Whether you have friends and family ready and able to help…
I am a financial data scientist. I think it is important that data scientists are highly specialized if they want to be effective in their careers. I run a business called Conlan Scientific out of Charlotte, NC where me and my team of financial data scientists tackle complicated machine learning problems for our clients. Quant trading is a gladiator’s arena of financial data science. Anyone can try it, but few succeed at it. I am sharing my top five list of math books that are essential to success in this field. I hope you enjoy.
Everyone knows what probability is, and we all understand how a coin flip works, but not everyone can explain the optimal betting strategies for a roulette table. We don’t study probability to understand the likelihood of events. We study probability to understand the expected outcomes of business processes that depend on those events.
In other words, this book won’t just teach you about probabilities, it will teach you about business strategies associated with those probabilities. It will help you answer a question like: How do I maximize the profit on this life insurance policy, given this set of survival probabilities? It isn’t just a likelihood question, it is a business question. I highly recommend that anyone studying probability does so through an actuarial lens.
This book covers the basic probability of distributions with an emphasis on applications from the areas of investments, insurance, and engineering. Written by a Fellow of the Casualty Actuarial Society and the Society of Actuaries with many years of experience as a university professor and industry practitioner, the book is suitable as a text for senior undergraduate and beginning graduate students in mathematics, statistics, actuarial science, finance, or engineering as well as a reference for practitioners in these fields. The book is particularly well suited for students preparing for professional exams, and for several years it has been recommended as…
I am a financial data scientist. I think it is important that data scientists are highly specialized if they want to be effective in their careers. I run a business called Conlan Scientific out of Charlotte, NC where me and my team of financial data scientists tackle complicated machine learning problems for our clients. Quant trading is a gladiator’s arena of financial data science. Anyone can try it, but few succeed at it. I am sharing my top five list of math books that are essential to success in this field. I hope you enjoy.
This is one of my favorite underappreciated statistics books of all time. Non-parametric statistics can be otherwise described as statistics without assumptions. The entire goal of this field of study is to prove X is greater than Y without making any assumptions about the underlying distributions of X or Y. The methods are different, and they require more data than other methods, but the learning journey is invaluable.
I personally believe that modern machine learning is simply the modeling section of the school of non-parametric statistics. Working through this book will give you a much deeper understanding of why tools like decision trees are so valuable. It will also to teach you to design rigorous numerical experiments on data sets that are beyond the help of classical statistics.
Guided by problems that frequently arise in actual practice, James Higgins' book presents a wide array of nonparametric methods of data analysis that researchers will find useful. It discusses a variety of nonparametric methods and, wherever possible, stresses the connection between methods. For instance, rank tests are introduced as special cases of permutation tests applied to ranks. The author provides coverage of topics not often found in nonparametric textbooks, including procedures for multivariate data, multiple regression, multi-factor analysis of variance, survival data, and curve smoothing. This truly modern approach teaches non-majors how to analyze and interpret data with nonparametric procedures…
I taught myself to code back in 1994 while working the graveyard shift as a geologist in the environmental industry. My job consisted of sitting in a chair during the dark hours of the night in a shopping center in Stockton, CA, watching another geologist take samples from wells in the parking lot. A friend of mine suggested I learn to code because I liked computers. I don’t mean to make this out to be a “it’s so simple anyone can do it!” You need to have a relentless drive to learn, which is why I wrote my book, The Imposter’s Handbook - as an active step to learning what I didn’t know I didn’t know.
You’ve heard of Einstein, Turing, Newton, and Hawking - but do you know who Claude Shannon is? Would you be surprised if I told you that he’s probably done more for our current way of life than all of the others combined? It’s true, and it’s unbelievable.
Claude Shannon was a quiet, quirky man who had what you might call The Most Genius Move of the last foreveryears: he took an obscure discipline of mathematics (Boolean Algebra) and applied it to electrical circuits, creating the digital circuit in the process. If you’ve ever wondered how 1s and 0s are turned into if statements and for loops - well here you go.
Oh, but that’s just the beginning. Dr. Shannon took things much further when he described how these 1s and 0s could be transmitted from point A to point B without loss of data. This was a big problem…
Winner of the Neumann Prize for the History of Mathematics
**Named a best book of the year by Bloomberg and Nature**
**'Best of 2017' by The Morning Sun**
"We owe Claude Shannon a lot, and Soni & Goodman’s book takes a big first step in paying that debt." —San Francisco Review of Books
"Soni and Goodman are at their best when they invoke the wonder an idea can instill. They summon the right level of awe while stopping short of hyperbole." —Financial Times
"Jimmy Soni and Rob Goodman make a convincing case for their subtitle while reminding us that Shannon…
My name is Daniel Robert McClure, and I am an Associate Professor of History at Fort Hays State University in Hays, Kansas. I teach U.S., African diaspora, and world history, and I specialize in cultural and economic history. I was originally drawn to “information” and “knowledge” because they form the ties between culture and economics, and I have been teaching history through “information” for about a decade. In 2024, I was finally able to teach a graduate course, “The Origins of the Knowledge Society,” out of which came the “5 books.”
This book starts in a similar historical location as Bod’s book but quickly moves through the nineteenth and twentieth centuries—settling into the “information theory” era established by Claude Shannon, Norbert Wiener, and others in the 1940s-1960s.
I love this book because it situates the intellectual climate leading to our current dystopia of information overload. Gleick’s teasing of chaos theory inevitably pushes the reader to explore his book on the subject from the 1980s: Chaos: Making a New Science (1987).
Winner of the Royal Society Winton Prize for Science Books 2012, the world's leading prize for popular science writing.
We live in the information age. But every era of history has had its own information revolution: the invention of writing, the composition of dictionaries, the creation of the charts that made navigation possible, the discovery of the electronic signal, the cracking of the genetic code.
In 'The Information' James Gleick tells the story of how human beings use, transmit and keep what they know. From African talking drums to Wikipedia, from Morse code to the 'bit', it is a fascinating…
