The blue secant joins the point at x and the point at x + h. Its slope is the average rate of change over that gap. Shrink h and the secant pivots into the terracotta tangent: that limiting slope is f prime of x, the instantaneous rate of change.

As you drag h toward zero the secant slope chases f prime of x and the gap collapses. That limit is the definition of the derivative; the closed form f prime is what the tangent is drawn from.

Now the input is a point in the plane and the output is a height. The gradient grad f is the vector of partial derivatives, the slope along x holding y fixed and the slope along y holding x fixed. It points in the direction of steepest increase and stands at a right angle to the contour through the point.

A single minimum at the origin. The gradient always points straight uphill, away from the center, and the level sets are nested ellipses it crosses at right angles.

The directional derivative is grad f dotted with the unit direction u, equal to |grad f| times the cosine of the angle between them. It peaks at +|grad f| when u points along the gradient, hits zero when u runs along the contour (perpendicular to the gradient), and bottoms at minus|grad f| pointing straight downhill. The bar above tracks that cosine.

A partial derivative is the Part 1 picture run on one axis: freeze every input but one, and you are back to a single curve whose slope is a plain derivative. The gradient collects those per-axis slopes into a vector. Because of that construction the gradient inherits two facts at once: its length is the steepest slope available at the point, and its direction is the way to climb fastest.

This is the engine under gradient-based learning. A model has thousands or millions of inputs instead of two, but the move is identical: compute the gradient of a loss, then step against it to go downhill. Every optimizer is a different rule for how far and how smoothly to take that step.