Coding examples
Sequence of k zeroes
- Naïve encoding takes k bits
- Run length takes about log k bits
1000 bits, 10 randomly placed 1’s, rest 0’s
- Posit a coin with bias 0.01
- Data encoding cost is (Shannon’s theorem):
-10 x lg 0.01 - 990 x lg 0.99 ? 81 bits << 1000 bits
Note that 10 x lg 100 ? 66 bits