blahut2 -- December94

First of all, I have to say a few words to my good friend, Dave Forney. This is a panel discussion so I suppose it is ok to have a debate and Dave did make the statement that convolutional codes are much more widely used than block codes. And I am usually not interested in taking one side or the other on this debate which is often heard and often silly. It is true, I will agree with Dave, that in applications to the additive white gaussian noise channel, convolutional codes are much more widely used than block codes. But if you look at all the channels that are out there in the world, then I think, by far it is true that block codes are much more widely used than convo- lutional codes. I was especially thinking of the compact disc player of which there are more of, by far, than there are modems. I am thinking of magnetic tape in disc drives and optical disc drives, all of which are using block codes. The closed caption annotation on American TV systems uses a block code to annotate TV programs for the hearing impaired. I know of applications to bar codes that use block codes. So if it is just a sheer matter of counting how many encoders and decoders there are, block codes are way out in front. One might also make the point that the additive white gaussian noise channel occurs much less frequently than other kinds of channels. Nonadditive nongaussian noise channels are everywhere, including storage, optimal communications, jamming, and multipath channels. Those are more difficult to understand and to model.

The question that was put to us is what things should infor- ma- tion theory be studying and what things should we not be study- ing. I was thinking of the fact that we have a large number of mathematicians or mathematically trained people in our group which leads to a positive result and also a nega- tive result. Both of which I would like to mention.

First of all, of all the engineering disciplines and even in the physics community, I think that we are the most rigorous and the most precise. When we say something is true, then it is true. We do not confuse proof with an opinion. And I think that is good. On the other hand, I notice that we sometimes become trapped on a problem when the problem is long since solved we continue to work on it and write papers. I mentioned Peter Elias before. I think Peter had a fictitous title of a paper he mentioned now and then whose title was something like "Detection of a Trianglularly Modulated Wave in the Presence of Two Sine Waves and Additive Gaussian Noise". In other words, a very artificial problem of no particular interest, but one that you could write a long series of papers.

As another negative example I might also think of decoding algorithms for a particular error correct- ing code, or a decoding algorithm for which there is only one code with which it can be used. We sometimes see papers like that too. Those are the kinds of papers we should not work on. Unless the reason we are working on those particular papers is that we are trying to open a door to understanding and by working on a small problem we might open a door into a whole wide area and learn new things. Then it is a legitimate problem to work on.

The reason I am mentioning that is because the question put to me by Sergio is what kinds of things I see that we should not be working on. I do not think we should be working on very arcane problems with no particular relevance to anything, just for the sake of having a topic to work on. There are other things that I think that we should work on that we are not, I would like to see some of us moving closer to problems of physics and computation.

In the case of communications we now understand very well the interaction between energy and communications. We have communication waveforms now that can transmit bits with energies measured in sub-micro-microjoules per bit. Meanwhile, in the area of computer science or computer architecture, I should say, or physics of com- putation, people are studying ways of doing a computation, how could one build a computer that does a million or billion word operations per second. How can we build one that minimizes the amount of energy expended?

There are conferences on this very subject right now. Conferences with names like the Physical Limits of Computation and people claiming that by ar- ranging gates in such and such a way or by doing this or that they can minimize the amount of energy a computer uses. Because after all in the area of computation now, the biggest issue in reducing the size or increasing the performance is reducing the amount of energy expended. Most computers are so small that they melt themselves or they are on the verge of melting themselves and any reduction in size or increase in the speed of the computer fails because the computer melts before it can do anything useful. So there is this question about how can we think about computation in a way that reduces the amount of energy expended and we in information theory, more than anyone else know how to organize communication waveform, and communication systems, so that we reduce the amount of energy expended. I think that we should have something more to say about the problem of computing in noise with minimum energy per gate operation, but we never interact with those people and I think it is a legitimate area.