jump to navigation

Review: The Theory of Information and Coding January 25, 2012

Posted by flashbuzzer in Research.
Tags: , , , ,

I recently finished “The Theory of Information and Coding” by R.J. McEliece. I originally used this book for a course taught by Dr. McEliece in the Fall 2002-03 quarter.

This book has one review on Amazon, so I hope to provide some additional insights for people who are considering obtaining the book.

In this book, the author provides the reader with a basic understanding of the fields of information theory and coding theory. He begins by presenting the concepts of entropy and mutual information, which are central to information theory. With these fundamental definitions in hand, he then presents the two main results of this area – Shannon’s channel coding and source coding theorems. After extending these results to the Gaussian case, he unifies them via a proof of the source-channel coding theorem. He then shifts gears and focuses on coding theory. He introduces several important types of codes, including linear codes – and some of their sub-classes, including cyclic codes, BCH codes and Reed-Solomon codes. The rest of the book contains a brief overview of convolutional codes and a discussion of variable-length source codes.

The author skillfully guides the reader through the information theory section of the text. By way of comparison, I had also read through one of the major texts in this area, Elements of Information Theory by Cover and Thomas; I found McEliece’s presentation of the fundamental information-theoretic concepts and results to be simpler and more intuitive. The author uses simple examples throughout the text to drive home key ideas; for example, he repeatedly discusses the (7,4) Hamming code – which happens to be a superb teaching tool. I also derived valuable insights from the presentation of the BCH decoding algorithm and the rigorous explanation of the efficacy of Huffman’s algorithm. In addition, I enjoyed the anecdotes that the author sprinkles throughout the text, as many of them arise from his long-standing association with JPL; thus, they bridge the gap between theory and practice.

My main issue with the text stems from the relatively large number of typos that I discovered. This is somewhat disappointing, as I read through its second edition. Also, the chapter on convolutional codes seems to be, technically speaking, rather light. This causes the section on coding theory to seem unbalanced, as the author skillfully presents many technical results that underlie the area of block coding. I would have preferred that the author extend the book to provide a more comprehensive treatment of convolutional codes, especially since they are more widely used in practice compared to block codes.

Overall I would strongly recommend this book to those who want a gentle introduction to the linked fields of information theory and coding theory.


No comments yet — be the first.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: