Expressive Capabilities of ANNs

Boolean functions:

• Every boolean function can be represented by network with single hidden layer
• but might require exponential (in number of inputs) hidden units

Continuous functions:

• Every bounded continuous function can be approximated with arbitrarily small error, by network with one hidden layer [Cybenko 1989; Hornik et al. 1989]

• Any function can be approximated to arbitrary accuracy by a network with two hidden layers [Cybenko 1988].