Nobody Wrote These Filters
When someone says a model learned something, what actually changed? A vision model shows you directly. Its first layer is a set of small grids of numbers that sit in the same space as pixels, so you can print them as pictures. Train one here on real photographs and watch sixteen grids of static turn into edge detectors nobody wrote, then slide one over a photo the model has never seen.
Nobody Wrote These Filters
When people say a model learned something, what changed inside it? For most models the answer is a shrug, but a vision model's first layer is small enough to look at. It is a set of tiny grids of numbers, and a grid of numbers can be drawn as a picture. Right now the sixteen below are static. Train the network on real photographs and watch them turn into edge detectors that nobody wrote.
The network is a pile of adjustable numbers called weights, 5,250 of them, all currently random. Training shows it a photo, checks whether its guess at what the photo contains was right, and nudges every weight a tiny step toward a better guess. The first layer is the first thing a photo passes through, and its weights are the grids you are about to see.
A filter is a small grid of numbers
Edge detection has existed since 1968, when Irwin Sobel and Gary Feldman picked nine numbers by hand and arranged them in a three by three grid. A grid like that is called a filter. To use it you slide it across the photograph, and at each position you multiply every grid number by the pixel underneath it and add the results into a single number, which becomes one pixel of the output. It is a weighted sum over a small patch, run at every position. Sobel's grid has a negative column down one side and a positive column down the other, so on flat areas the sum cancels to nothing and wherever brightness changes it spikes. Move over the photograph to see one position at a time.
Loading photographs…
Terracotta where the sum comes out positive, blue where it comes out negative, pale where the grid found nothing. For a slower walk through the multiply and add, see Convolution: Kernels on an Image.
Train it and watch the static resolve
The network gets 8,000 photographs at 32 by 32 pixels, each labelled with one of ten things: cat, ship, truck, frog and so on. It sees brightness only, no colour. Its one job is to guess the label, and after each guess it adjusts its weights to be a little less wrong. That measure of wrongness is called the loss, and nothing in it mentions edges, or orientation, or the word filter.
Each tile is one filter's 49 numbers drawn as a picture, shaded the same way as the Sobel numbers above: dark for negative, light for positive, grey for zero. A blue tick means the tile now matches an oriented edge, and the tick lies along that edge. Click a tile to inspect it.
Terracotta where the photograph matches the filter's light side, blue where it matches its dark side, pale where it matches neither. This is the same slide, multiply and add as the Sobel demo, only the nine numbers have been replaced by these forty nine.
Accuracy settles somewhere around forty percent, four times better than guessing and nowhere near a real vision model. The filters are the point, not the score.
Measure how edge-like it is
Clicking a tile above picks the filter shown here. To call a tile an edge detector without eyeballing it, each filter is compared with ideal edges at 36 angles, and the closest match becomes its score: 1 is a perfect edge and 0 is no resemblance. Out of 192 freshly initialised random filters, not one cleared 0.6.
Neuroscientists run this same test on cells in the visual cortex: show a striped patch at every angle in turn and record how hard the cell answers. A trained filter usually gives one hump, which means it has picked an orientation and mostly ignores the rest. The dashed line is the same filter before training, when it answered weakly to everything and strongly to nothing. Each line is drawn against its own peak so the shapes can be compared, and the peaks are printed underneath because the trained filter also answers several times harder.
So what does it mean that a model learned something
It means the numbers moved. Nobody told the network what an edge is; the only feedback was whether its guess about the photo was right, and the weights drifted, one small nudge at a time, until the first layer held something useful. Edges are useful because they are what photographs are made of: outlines, textures, the boundary of a shadow. This network has 5,250 weights against 8,192,000 pixel values in its training photos, so it cannot store the photos and has to keep what they share. A much bigger network has room to store far more, yet its first layer still learns oriented edge filters much like those found in the cells of an animal's visual cortex, because that structure comes from photographs, not from the size of the budget.
The same goes for grammar, or arithmetic, or anything else a language model appears to know. No rule was written down anywhere. Numbers were nudged, one gradient step at a time (a tiny change to every weight, in the direction that shrinks the loss), until they encoded something general enough to work on inputs nobody had shown the model. Vision hands you the receipt as a picture, because a first-layer filter lives in pixel space and can be printed. The weights of a language model, a design called a transformer, map one abstract space to another, so printing them raw gives you static, and finding what they learned takes other instruments: the geometry of an embedding table (how words are placed as points in space), the pattern an attention head settles into (which earlier words a word looks at), or the plain fact that a working model is a file of numbers you can corrupt.