An ill-conditioned bowl, two hundred times steeper across one axis than along the other. At the default rate, which sits right at SGD's stability edge for the steep axis, plain SGD oscillates and stalls while the others descend cleanly. One shared rate cannot serve both axes, which is exactly what per-dimension scaling fixes.

Adam's second moment uses beta2 = 0.999, eps = 1e-8. Changing any slider relaunches the race from the current start so the comparison stays fair.

On the ravine, let it run: Momentum, RMSProp, and Adam all reach a far lower loss than plain SGD in the same number of steps. At this rate SGD sits right at its stability edge for the steep axis, so it oscillates across the valley and its loss plateaus while the others slide down to the floor.

Plain SGD takes a fixed fraction of the raw gradient, so in a ravine it must keep the step small enough to not blow up on the steep axis, which leaves it crawling on the flat one. Momentum accumulates a velocity that averages out the side-to-side gradient and keeps the consistent down-valley component, so it powers through. RMSProp divides each axis by the running size of its own gradients, shrinking the step where gradients are large and growing it where they are small, which evens out the conditioning. Adam combines both: a momentum-like first moment for direction and an RMSProp-like second moment for per-axis scale, with a bias correction so the early steps are not damped toward zero.