The individual redundancy of a source code, relative to the actual
source, for sequence 
is defined as
The base of the logarithm is assumed to be 2 here. For the expected redundancy we obtain
i.e. the difference between the expected codeword length 
and the entropy 
of the source sequences.
The smallest possible expected redundancy is 0.
Proof:Rewrite
where 
. The first term on the right hand side
is a divergence and can not be negative (see [1] page 26).
The second term is non-negative by Kraft's inequality (1).
Note that we achieve 
only if the codeword length 
for all .
Therefore we call such codeword lengths ideal.
It is our intention to design codes that approach these ideal codeword
lengths as closely as possible.
Example:Let T=2. Consider the code 
and a
memoryless source with 
. The table below lists the
individual redundancies for this code.
The expected redundancy 
.