Step the forward pass to push this example through the network, then step the backward pass to watch the chain rule assign a gradient to every weight. Or skip ahead and run a gradient-descent update.

Larger steps descend faster but can overshoot and oscillate. Too small and the loss crawls. Watch the curve below react when you change it mid-training.

A gradient is the direction of steepest increase in the loss. Each weight's gradient answers one question: if I nudge this weight up a little, does the loss rise or fall, and how sharply? Backprop computes all of those answers in a single backward sweep by reusing the upstream gradient at each layer, which is why it is far cheaper than poking each weight one at a time. To shrink the loss we move every weight a small step in the opposite direction of its gradient, scaled by the learning rate. Repeat over the four examples and the network bends its decision boundary until XOR comes apart.