In this Section, only a fraction of statistical applications of IT could be covered. For others, and for more information about those only tangentially mentioned here, let me refer to the excellent survey Barron 1997.
Let me just mention one field that jointly belongs to IT and statistics, and obviously requires methods of both disciplines: hypothesis testing and estimation based on remote observations, subject to rate constraints on permissible communication. Works about hypothesis testing and estimation problems, respectively, with communication constraints, include Ahlswede and Csiszár 1986, Han 1987, Shalaby and Papamarcou 1992, respectively Zhang and Berger 1988, Ahlswede and Burnashev 1990, Han and Amari 1995.
truecm
1cm Ahlswede, R. and Burnashev, M. (1990) ``On minimax estimation in presence of side information about remote data,'' Ann. Statist., vol.18, pp.141-171.
1cm Ahlswede, R. and Csiszár, I. (1986) ``Hypothesis testing with communication constraints,'' IEEE Trans. IT, vol.32, pp.533-542.
1cm Ahlswede, R., Gács, P. and Körner, J. (1976) ``Bounds on conditional probabilities with applications in multiuser communication,'' Z. Wahrscheinlichkeitsth. verw. Gebiete, vol.34, pp.157-177.
1cm Akaike, H. (1973) ``Information theory and an extension of the maximum likelihood principle,'' Second Int'l Symp. Inform. Theory, pp.267-281, B. N. Petrov and F. Csáki, eds., Akadémiai Kiadó, Budapest.
1cm Arimoto, S. (1972) ``An algorithm for computing the capacity of discrete memoryless channels,'' IEEE Trans. IT, Vol.18, pp.14-20.
1cm Barron, A. R. (1986) ``Entropy and the central limit theorem,'' Ann. Probab., vol.14, pp.336-342.
1cm Barron, A. R. (1997) ``Information theory in probability, statistics, learning, and neural nets,'' Manuscript.
1cm Bernardo, J. M. (1979) ``Reference posterior for Bayesian inference,'' J. Roy. Statist. Soc. B, vol.41, pp.113-147.
1cm Blahut, R. (1972) ``Computation of channel capacity and rate-distortion functions,'' IEEE Trans. IT, vol.18, pp.460-473.
1cm Byrne, C. (1993) ``Iterative image reconstruction algorithms based on cross-entropy minimization,'' IEEE Trans. Image Processing, vol.2, pp.96-103.
1cm Byrne, C. (1996) ``Alternating minimization, generalized orthogonality and Pythagorean identities in iterative image reconstruction,'' SIAM J. Optimization, submitted.
1cm Chernoff, H. (1952) ``A measure of asymptotic efficiency for tests of a hypothesis based on a sum of observations,'' Ann. Math. Statist., vol.23, pp. 493-507.
1cm Clarke, B. and Barron, A. (1994), ``Jeffreys' prior is asymptotically least favorable under entropy risk,'' J. Statist. Planning and Inference, vol.41, pp. 37-60.
1cm Cover, T. (1984) ``An algorithm for maximizing expected log investment return,'' IEEE Trans. IT, vol.30, pp. 369-373.
1cm Csiszár, I. (1965) ``A note on limiting distributions on topological groups,'' Publ. Math. Inst. Hungar. Acad. Sci., vol.9, pp.595-599.
1cm Csiszár, I. (1975) ``I-divergence geometry of probability distributions and minimization problems,'' Ann. Probab., vol.3, pp.146-158.
1cm Csiszár, I. (1984) ``Sanov property, generalized I-projections, and a conditional limit theorem,'' Ann. Probab., vol.12, pp.768-793.
1cm Csiszár, I. (1989) ``A geometric interpretation of Darroch and Ratcliff's generalized iterative scaling,'' Ann. Statist., vol.17, pp.1409-1413.
1cm Csiszár, I. and Körner, J. (1981) Information Theory: Coding Theorems for Discrete Memoryless Systems, Academic.
1cm Csiszár, I. and Tusnády, G. (1984) ``Information geometry and alternating minimization procedures,'' Statistics and Decisions, Suppl.1, pp.205-237.
1cm Darroch, J. N. and Ratcliff, D. (1972) ``Generalized iterative scaling for log-linear models,'' Ann. Math. Statist., vol.43, pp.1470-1480.
1cm Della Pietra, S., Della Pietra, V. and Lafferty, J. (1997), ``Bregman distances, iterative scaling, and auxiliary functions,'' Manuscript.
1cm Dembo, A. and Zeitouni, O. (1993), Large Deviations Techniques and Applications, Jones and Bartlett.
1cm Deming, W. E. and Stephan, F. F. (1943) ``On a least squares adjustment of a sampled frequency table when the expected marginal totals are known,'' Ann. Math. Statist., vol.11, pp.427-444.
1cm Dempster, A., Laird, N. and Rubin, D. (1977) ``Maximum likelihood from incomplete data via the EM algorithm,'' J. Roy. Statist. Soc., B, vol.39, pp.1-38.
1cm Efroimovich, S. Y. and Pinsker, M. S. (1982) ``Estimation of square-integrable probability density of a random variable'' (in Russian), Probl. Pered. Inform., vol.18, no.3, pp.19-38.
1cm Fisher, R. A. (1925) ``Theory of statistical estimation,'' Proc. Camb. Phil. Soc., vol.22, pp.700-725.
1cm Fritz, J. (1973) ``An information theoretic proof of limit theorems for reversible Markov processes,'' Trans. Sixth Prague Conference on Inform. Theory etc., pp.183-197, Academia, Prague.
1cm Good, I. J. (1950) Probability and the Weighing of Evidence, Griffin, London.
1cm Groeneboom, P., Oosterhoff, J. and Ruymgaart, F. H. (1979) ``Large deviation theorems for empirical probability measures,'' Ann. Probab., vol.7, pp.553-586.
1cm Hájek, J. (1958) ``On a property of normal distributions of any stochastic process'' (in Russian), Czechoslovak Math. J., vol.8, pp.610-617.
1cm Han, T. S. (1987) ``Hypothesis testing with multiterminal data compression,'' IEEE Trans. IT, vol.33, pp.759-772.
1cm Han, T. S. and Amari, S. (1995) ``Parameter estimation with multiterminal data compression,'' IEEE Trans. IT, vol.41, pp.1802-1833.
1cm Hasminskii, R. Z. (1978) ``A lower bound on the risks of nonparametric estimates of densities in the uniform metric,'' Theory Probab. Appl., vol.23, pp.794-796.
1cm Hoeffding, W. (1965) ``Asymptotically optimal tests for multinomial distributions,'' Ann. Math. Statist., vol.36, pp.369-400.
1cm Ibragimov, I. A. and Hasminskii, R. Z. (1973), ``On the information in a sample about a parameter,'' Second Int'l Symp. Inform. Theory, pp. 295-309, B. N. Petrov and F. Csáki, eds., Akadémiai Kiadó, Budapest.
1cm Ibragimov, I. A. and Hasminskii, R. Z. (1982) ``Bounds for the risks of non-parametric regression estimates,'' Theory Probab. Appl., vol.27, pp. 84-99.
1cm Ireland, C. T. and Kullback, S. (1968) ``Contingency tables with given marginals,'' Biometrika, vol.55, pp.179-188.
1cm Kendall, D. G. (1963) ``Information theory and the limit theorem for Markov chains and processes with a countable infinity of states,'' Ann. Inst. Stat. Math., vol.15, pp.137-143.
1cm Kolmogorov, A. N. (1958) ``A new invariant for transitive dynamical systems'' (in Russian), Dokl. A.N.SSSR, vol.119, pp.861-864.
1cm Kruithof, R. (1937) ``Telefoonverkeersrekening,'' De Ingenieur, vol.52, pp.E15-E25.
1cm Kullback, S. (1959) Information Theory and Statistics, Wiley.
1cm Kullback, S. (1968) ``Probability densities with given marginals,'' Ann. Math. Statist., vol.39, pp. 1236-1243.
1cm Kullback, S. and Leibler, R. A. (1951) ``On information and sufficiency,'' Ann. Math. Statist., vol.22, pp.79-86.
1cm Linnik, Yu. V. (1959) ``An information theoretic proof of the central limit theorem on Lindeberg conditions'' (in Russian), Teor. Veroyat. Primen., vol.4, pp. 311-321.
1cm Margulis, G. A. (1974) ``Probabilistic characteristics of graphs with large connectivity'' (in Russian), Probl. Pered. Inform., vol.10, no.2, pp.101-108.
1cm Marton, K. (1986) ``A simple proof of the blowing up lemma,'' IEEE Trans. IT, vol.32, pp.445-446.
1cm
Marton, K. (1996) ``Bounding 
-distance
by informational divergence: a method to prove measure
concentration,'' Ann. Probab., vol.24, pp.857-866.
1cm Marton, K. and Shields, P. (1994) ``The positive divergence and blowing up properties,'' Israeli J. Math., vol.86, pp.331-348.
1cm Matus, F. (1997) ``On iterated averages of I-projections,'' Ann. Statist., submitted.
1cm Pinsker, M. S. (1972), ``Information contained in observations, and asymptotically sufficient statistics'' (in Russian), Probl. Pered. Inform., vol.8, pp.45-61.
1cm Rényi, A. (1961) ``On measures of entropy and information,'' Proc. Fourth Berkeley Symposium Math. Statist. Probab., vol.1, pp.547-561, Univ. Calif. Press.
1cm Rényi, A. (1969) ``On some problems of statistics from the point of view of information theory,'' Proc. Coll. Inform. Theory, pp.343-357, J. Bolyai Math. Soc., Budapest.
1cm Rissanen, J. (1978) ``Modeling by shortest data description,'' Automatica, vol.14, pp. 465-471.
1cm Rissanen, J. (1989) Stochastic Complexity in Statistical Inquiry, World Scientific.
1cm Rüschendorf, L. (1995) ``Convergence of the iterative proportional fitting procedure,'' Ann. Statist., vol.23, pp.1160-1174.
1cm Sanov, I. N. (1957) ``On the probability of large deviations of random variables'' (in Russian), Math. Sbornik, vol.42, pp.11-44.
1cm Shalaby, H. and Papamarcou, A. (1992) ``Multiterminal detection with zero-rate data compression,'' IEEE Trans. IT, vol.38, pp.254-267.
1cm Shannon, C. E. (1956) ``The bandwagon,'' IRE Trans. IT, vol.2, p.3.
1cm Shields, P. C. (1996) The Ergodic Theory of Discrete Sample Paths, Graduate Studies in Math., vol.13, Amer. Math. Soc.
1cm Talagrand, M. (1995) ``Concentration of measure and isoperimetric inequalities in product spaces,'' Publ. Math. IHES, vol.81, pp.73-205.
1cm Talagrand, M. (1996) ``A new look at independence,'' Special Invited Paper, Ann. Probab., vol.24, pp.1-34.
1cm Tusnády, G. (1977) ``On asymptotically optimal tests,'' Ann. Statist., vol.5, pp.385-393.
1cm Wald, A. (1947) Sequential Analysis, Wiley.
1cm Yang, Y. and Barron, A. R. (1997) ``Information-theoretic determination of minimax rates of convergence,'' to appear in Ann. Statist.
1cm Yu, B. (1995) ``Assouad, Fano, and Le Cam,'' to appear in Festschrift in honor of Lucien Le Cam.
1cm Zhang, Z. and Berger, T. (1988) ``Estimation via compressed information,'' IEEE Trans. IT, vol.34, pp.198-211.