**Batch mode** Gradient Descent:

Do until satisfied

- 1.
- Compute the gradient
- 2.

**Incremental mode** Gradient Descent:

Do until satisfied

- For each training example
*d*in*D*- 1.
- Compute the gradient
- 2.

*Incremental Gradient Descent* can approximate *Batch Gradient
Descent* arbitrarily closely if
made small enough