Decision Boundary Zoo
A straight line cannot separate two moons. Train four classifiers on the same tangled data and watch the linear model fail while kNN, an RBF SVM, and a small neural net curve around it.
Decision Boundary Zoo
A classifier carves the plane into regions, one per class; the seam between them is the decision boundary. Pick a dataset and a model, and the boundary you see is traced by classifying a fine grid of points after the model trains. Watch where a straight line is enough and where it bends or gives up.
Two interleaving half circles. No straight line can split them cleanly, but a curve threading between the arcs can. A linear model gets the gist and stalls in the high eighties, while the nonlinear models climb past it.
Linear. The boundary is always a single straight line, so it can only solve problems where one class sits cleanly on one side.
A dataset is linearly separable when a single straight line can put each class on its own side. The blobs are; the moons, circles, and XOR are not. Logistic regression can only draw that one straight line, so on the nonlinear sets it stalls: near chance on circles and XOR, stuck in the high eighties on moons no matter how long you train it. The shaded regions make the failure visible. The line simply cannot bend.
The other three buy nonlinearity in different currencies. kNN pays with memory and locality: it keeps every point and lets the nearest neighbors vote, so the boundary follows the data and frays into islands when k is small. The kernel SVM pays with dimensions: it lifts the two features into a high-dimensional random feature space where a straight max-margin cut becomes a smooth curve back in the plane, and gamma sets how tightly that curve hugs the points. The neural net pays with learned weights: stacked tanh layers bend the input space until the classes come apart, and as you add hidden units and epochs the boundary tightens, eventually wrapping around individual noisy points, which is overfitting you can watch happen.