Entropy(S) = expected number of bits needed to encode class (
of randomly drawn member of S (under the optimal, shortest-length
- Information theory: optimal length code assigns
message having probability p.
So, expected number of bits to encode
of random member