Occam's Razor

Why prefer short hypotheses?

Argument in favor:

  $\mbox{$\bullet$}$

Fewer short hyps. than long hyps.

$\rightarrow$

a short hyp that fits data unlikely to be coincidence

$\rightarrow$

a long hyp that fits data might be coincidence

Argument opposed:

• There are many ways to define small sets of hyps
• e.g., all trees with a prime number of nodes that use attributes beginning with ``Z''
• What's so special about small sets based on size of hypothesis??