This blog post looks at variants of gradient descent and the algorithms that are commonly used to optimize them.

Source: An overview of gradient descent optimization algorithms