verdu1 -- December94
People who claim information theory is dead are saying, in a way, "I wish information theory were dead". It is rather amusing to hear how some people outside the field view in- formation theory; some "definitions" you may have heard be- fore are: "Information Theory is Entropy", "Information Theory is Jensen's inequality", "Information Theory is a one-theorem theory" (attributed to a famous Stanford probabilist), and "Information Theory is the AEP". If either of those parodies were halfway correct, research in information theory would surely be dead by now.

I will focus my answer on Shannon theory as it applies to the search for fundamental limits in source and channel coding. By their very nature, the answers to those questions are technology-independent, which does not mean that in our work we disregard the technology of the day. On the con- trary, the technological state-of-the-art has always been the main driving force in selecting research problems, and moreover, technology is on our side because it enables us to approach those fundamen- tal limits closer and closer as time progresses.

Some people question whether information theory has anything practically relevant to say in modern communication systems. It would be wrong to assume that technological advances rapidly render any channel model obsolete, and that therefore it is irrelevant to try to push as much information through a channel as possible, or even to find its capacity. Consider, for example, wireless, deep-space and telephone chan- nels. Indeed, channels whose bandwidth and/or signal-to- noise ratio are precious resources are the norm rather than the exception.

Were we not able to come close to reaching capacity or source entropy rates with reasonable complexity, then the practical relevance of Shannon theory would indeed be debatable. However, the practical application of our field has produced stellar success stories such as the Ziv-Lempel algorithm, coding in space communication ("a marriage made in heaven" in the words of Jim Massey), and the telephone chan- nel, where after fifty years, the limits predicted by Shan- non have been almost reached by commercial modems. That lesson is and will be valid in many other examples. Maybe the day will come when a software package will enable the engineer to closely approach the capacity of almost any channel with the technology of the day. Admittedly, I am afraid it is us who will be dead when that day arrives!