Explorable explanations
Interactive artifacts that make complicated ideas click.
A small, growing collection of things I'm trying to understand, rebuilt as something you can poke at. Each piece started as an explanation and became a sandbox.
Streaming a 26B MoE From SSD on a Jetson
A single request on an 8 GB Jetson Orin Nano, with the model's experts coming off SSD mid-inference. Scrub the run behind the blog post's GIF and watch GPU load spike as each expert read lands, or flip to the bench numbers.
Open →Diffusion Language Models: All the Words at Once
A normal LLM writes one token per forward pass because each word has to wait for the last. A diffusion language model drafts the whole line and firms it up over a few passes. Race both decode orders on the same tiny model trained for this page, dissolve a sentence into masks, give the denoiser fewer steps and read the damage, then pin both ends of a story and let the middle fill itself in.
Open →A Global Workspace Inside the Model
Nobody told the model the author was British. It worked that out from a few spellings and then started spelling colour, not color, on its own. With the paper's lens I can read that unstated decision forming and steering the output. It is a small window onto the paper's larger claim, that language models hold a shared, readable set of active ideas. The readouts are from a 4B model I ran myself.
Open →The Reasoning Step It Never Says Out Loud
I asked a model for the currency of the country shaped like a boot and it answered euro. To get there it first has to decide the country is Italy, but it never writes Italy. That step happens silently, inside. Using the paper's lens I caught Italy surfacing mid-network before it hands off to euro. The readouts are from a 4B model I ran myself.
Open →The Word It Has Already Chosen
I misspelled langauge and the model read it as language anyway. I set up a rhyme and it locked the last word in long before writing it. Decoding the model's internal guess at each layer with the paper's lens, I could see the correction and the rhyme were settled well before the output. The readouts are from a 4B model I ran myself.
Open →Watch a Language Model Do Arithmetic
I gave a language model (2 + 3) * 4 and it wrote 20 without running a calculator. Using the paper's lens I read the model's half-formed answer out of each layer and watched it reach the sub-sum 5 partway up, then twenty near the top. The readouts are from a 4B model I ran myself.
Open →EVOKE: Benchmark Results
Can an LLM still answer questions about content it evicted from its KV cache to save memory? EVOKE saves the exact K/V tensors instead of throwing them away and recovers them with no recompute. These results test that against heuristic eviction and a retrieval-based baseline, across model architectures, memory budgets, and a live agent session.
Open →Inside an LLM
A real trained GPT, small enough to read, runs in your browser: watch a prompt become matrix math become the next character. Then corrupt a weight and see for yourself that the model is just a file of numbers.
Open →Attention, From the Ground Up
The word bank means a riverside in one sentence and a lender in another, yet the model stores just one meaning for it. Attention fixes that by letting each word borrow meaning from its neighbors: drag the weights around, read a hand-built head doing grammar, then watch a real one learn where to look.
Open →EVOKE: Evicting and Recovering the KV Cache
An agent session outgrows its GPU memory budget fast. Watch every cache block earn a relevance score from how well it still matches the task in focus, see the lowest scorers evicted to host RAM, then spliced back by content identity when the history is re-sent, so almost nothing is recomputed.
Open →Recurrent Networks, End to End
A plain RNN forgets fast. Train one against an LSTM on a long-range task and watch the vanilla net's gradient die through time while the LSTM's cell-state path keeps memory alive.
Open →Inside an LSTM Cell
An LSTM remembers across gaps where a plain RNN forgets. Open one cell, run it on a sequence, and watch the forget, input, and output gates carry memory along the cell state's near-identity highway.
Open →Bayesian vs Frequentist
One coin, one question: what is its bias? Answer it two ways on the same flips, a confidence interval that earns its 95% over repeated samples and a Bayesian posterior that says how probable each bias is.
Open →The Gradient and the Derivative
A derivative is a secant line in the limit, and a gradient is two of them at once. Drag a point to read the tangent on a curve and the steepest-ascent vector on a contour map.
Open →Matrix as a Linear Transformation
A 2x2 matrix bends the whole plane while keeping every line straight. Drag where the basis vectors land and watch the grid follow, the determinant as signed area, and the eigenvectors as the directions that refuse to turn.
Open →Activation Functions, Curve by Curve
Sigmoid and tanh flatten at the edges, and their gradient dies there. Drag a point across the curves to compare slopes, and watch the ReLU family keep the gradient alive where the others go silent.
Open →Decision Boundary Zoo
A straight line cannot separate two moons. Train four classifiers on the same tangled data and watch the linear model fail while kNN, an RBF SVM, and a small neural net curve around it.
Open →PCA vs t-SNE: What Each One Lies About
Both flatten high-dimensional data into a picture, and both lie. PCA keeps the global shape but cannot unbend a curve; t-SNE draws clean clusters whose sizes and spacing mean nothing. Run both and see where each one cheats.
Open →Word Embeddings and Analogy Arithmetic
King minus man plus woman lands on queen, and it sounds like a trick. Place words in a space where meaning is direction, then watch the arithmetic fall out of the geometry.
Open →Vanishing and Exploding Gradients
Stack enough layers and the gradient either fades to nothing or explodes. Tune depth, activation, and weight scale, watch the per-layer gradient norm decay or blow up, then see it happen through time in an RNN.
Open →IEEE 754 Floating Point
0.1 plus 0.2 is not 0.3, and a float explains why. Toggle the 32 bits of sign, exponent, and mantissa, type a decimal, and watch the nearest value your hardware can store snap into place.
Open →Dot Product, Cosine, and Projection
Two arrows from the origin, and one number that says how much they agree. Drag them apart and watch the dot product fade to zero where they cross at a right angle, then flip negative as they swing opposite.
Open →Two's Complement: Signed vs Unsigned
The same bits mean 200 unsigned and -56 signed. Toggle them, walk the number wheel where 127 wraps to -128, and negate a value by flipping every bit and adding one.
Open →Q-learning on a Gridworld
An agent that knows nothing learns a path from reward alone. Watch value fill backward from the goal and the greedy arrows snap into a route, then see how cutting off exploration too soon traps it on a worse one.
Open →The Interactive Byte
Toggle eight bits and watch one number speak binary, decimal, hex, octal, and ASCII at once. Then play with AND, OR, XOR, NOT, and shifts computed column by column.
Open →Solve vs Verify: P vs NP Made Concrete
Checking an answer can be instant while finding one takes forever. Watch a DPLL solver grind through an exponential search tree while a checker verifies any solution in a glance: the gap behind P vs NP.
Open →Huffman Coding
Frequent letters deserve shorter codes. Type a string, merge the two rarest symbols again and again, and watch a Huffman tree grow whose codeword lengths land within a bit of the entropy floor.
Open →Dynamic Programming: the Explosion and the Collapse
Naive fib(20) makes 21,891 calls because the same subproblems keep coming back. Hand the recursion a cache and watch the tree collapse to 39 calls.
Open →Optimizer Race
Drop four optimizers on the same loss surface from the same point and race them. Watch momentum carry through a ravine and Adam rescale each axis while plain SGD crawls or oscillates.
Open →Boids: Flocking From Local Rules
No leader, no plan. Each boid watches only its nearby neighbors and obeys three steering rules. Dial separation, alignment, and cohesion and watch a flock cohere or fall apart.
Open →Barnes-Hut: Gravity in n log n
Hundreds of bodies pull on each other, but a quadtree lets distant clumps act as one mass, turning an n squared force sum into n log n. Tune the opening angle and watch accuracy trade against speed.
Open →Tokenization and Byte Pair Encoding
A language model never sees your letters, it sees tokens, chunks of text that fall between letters and words. Train a real BPE merge table on a small corpus, then watch your own text break into the pieces a model actually reads.
Open →Wave Function Collapse
Every cell starts as every possible tile at once, then collapses one at a time. Each choice ripples through its neighbors as a constraint wave until the whole map agrees, and a bad pick forces a backtrack.
Open →The Central Limit Theorem
Start from a lopsided source, a hard skew or a U-shape, draw samples and average each batch, then watch the histogram of those averages climb into a bell no matter how lumpy the source was.
Open →PageRank by Random Walk
One random surfer, clicking forever, ranks a whole web. Build a graph and watch a surfer's visit frequencies converge to the same PageRank vector that power iteration computes.
Open →Kalman Filter: Tracking Through Noise
Track a moving target through noisy measurements. Watch the predict step grow uncertainty and the update step shrink it, and drag the noise sliders to see the covariance ellipse breathe.
Open →Hash Tables and Collisions
A dictionary lookup feels instant, yet two different keys can land in the same bucket. Insert keys, watch the hash pick a slot, and see what the table does when slots collide and when it runs out of room.
Open →K-Means: Finding Clusters by Hand
Drop points on a plane and watch Lloyd's algorithm chase the cluster centers, assign, recenter, repeat. The same data can settle differently each run, the local minima that make k-means restart-sensitive.
Open →Convolution: Kernels on an Image
Edge detection, blur, and sharpen are all the same nine-number trick. Slide a 3x3 kernel across an image and watch the multiply-and-sum behind each output pixel, the operation a CNN learns on its own.
Open →Raft: Leader Election & Replication
Five nodes have to agree on one log, even when servers crash. Kill the leader and watch randomized timeouts elect a new one, replicate to a majority, and recover without losing a committed entry.
Open →Next-Token Prediction & Sampling
A language model writes one token at a time, drawing each from a probability distribution and feeding its own choice back in. Turn the temperature, top-k, and top-p knobs and watch the same distribution reshape under your hands.
Open →Bloom Filters
A bloom filter can say a key is present when it never was. Insert keys, watch their bits light up across k hash probes, and catch the false positive the moment it happens.
Open →Monty Hall
Once the host opens a goat door, switching feels like it cannot matter. Play the game, then run thousands of rounds and watch staying settle at 1/3 while switching settles at 2/3.
Open →Fourier: Drawing with Circles
Any closed drawing is just a stack of spinning circles. A Fourier transform finds them, and adding one circle at a time sharpens a crude loop into the shape you drew.
Open →Diffie-Hellman Key Exchange
Two strangers end up holding the same secret number, yet that number is never sent across the wire. Watch them build it in the open while an eavesdropper reads every message and still cannot follow.
Open →Consistent Hashing
Add or remove a server and almost nothing should move. Place nodes and keys on a hash ring and watch why only a thin slice of keys shifts, while mod-N reshuffles nearly everything.
Open →Diffusion: Noise and Denoise
Destroy a shape into pure noise, then rebuild it. Watch a point cloud dissolve under the forward process, then a score-based sampler walk it back to the data one small step at a time.
Open →Semantic Search: Meaning, Not Keywords
You search for will it rain and the best answer says clear skies tomorrow, never once using your words. Watch meaning win where matching letters cannot.
Open →Backpropagation, by Hand
A tiny network learns XOR while you watch. Step the forward pass neuron by neuron, then trace the gradient backward edge by edge as the chain rule assembles itself.
Open →UTF-8 and Unicode
One emoji can be four bytes, and string length lies about it. Type any text and watch it split into graphemes you see, the code points Unicode assigns, and the UTF-8 bytes that get stored.
Open →Pathfinding: Dijkstra & A*
Dijkstra floods a grid in every direction while A* aims straight at the goal. Race them on the same maze and see why a good heuristic visits far fewer cells for the same shortest path.
Open →House Prices with MCMC
Forty houses, four unknown weights, and a Markov chain that wanders the whole posterior. Watch it converge, then predict a price not as one number but as a full distribution.
Open →Bezier Curves and de Casteljau
Drag four dots and a smooth curve bends to follow. The trick is repeated averaging: split each line at the same fraction, then split the splits, until one point is left to trace.
Open →