Sigmoid Unit

$\sigma(x)$ is the sigmoid function

$$\frac{1}{1 + e^{-x}}$$

Nice property: $\frac{d \sigma(x)}{dx} = \sigma(x) (1 - \sigma(x))$

We can derive gradient descent rules to train

• One sigmoid unit

• Multilayer networks of sigmoid units $\rightarrow$ Backpropagation