The Mathematical Theory of Communication

While studying computer networks, Claude Shannon did something pretty impressive. He reformulated the majority of classical statistics from scratch using the language and concepts of computer science. 

Statistical noise? There’s a new word for that; it’s called entropy. Also, it turns out it is a good thing, not a bad thing because entropy is equal to the information content or a data set. Tired of minimizing the squared error of everything? That’s fine, minimize the log of its likelihood instead. It does the same thing. This book challenges the assumptions of classical statistics in a way that fits neatly… show more.

This is really two books, strapped together. The first book by Weaver is an informal introduction to the ideas implicit in the second book by Shannon. Shannon’s book is naturally quite dated in its use of language (e.g. uncertainty is pronounced equivocation), and in its references to ‘current’ technology (e.g. PCM). Despite these caveats, it is still a surprisingly accessible book. Finally, because information theory was developed almost exclusively by Claude Shannon, reading the theory explained in his own words gives some insight into how on Earth he managed to come up with such a radical set of ideas.