Backpropagation Algorithm

Initialize all weights to small random numbers.

Until satisfied, Do

• For each training example, Do

  1.

  Input the training example to the network and compute the network outputs

  2.

  For each output unit k

  
  

  $$\delta_{k} \leftarrow o_{k}(1-o_{k})(t_{k}-o_{k})$$

  
  

  3.

  For each hidden unit h

  
  

  $$\delta_{h} \leftarrow o_{h}(1-o_{h})\sum_{k \in outputs}w_{h,k}\delta_{k}$$

  
  

  4.

  Update each network weight w_i,j
  

  $$w_{i,j} \leftarrow w_{i,j} + \Delta w_{i,j}$$

  
  

  where

  
  

  $$\Delta w_{i,j} = \eta \delta_{j} x_{i,j}$$