There are many other applications of IT to probability that can not be covered here.
Let us just mention that various limit theorems of
probability theory have been given information theoretic
proofs. These include:
Central limit theorem (Linnik 1959, Barron 1986);
Ergodicity of Markov chains (Rényi 1961, Kendall 1963,
Fritz 1973);
Limit theorem for the convolution powers of a PD on
a topological group (Csiszár 1965).
A promising recent idea is to prove bounds for recurrence and matching problems, utilizing the non-existence of codes beating the entropy bound (Shields 1996, Section II.5).