What you’re watching
Imagine a student cramming for a test. They’re handed a third of the answers to a huge addition table, and with enough time they can memorize every one. Ask about those and they’re perfect. Ask about anything else and they’re guessing, because they never learned addition.
That’s the setup, with a small neural network instead of a student. It learns addition on a clock with 113 hours, so 100 + 20 wraps around to 7. There are 12,769 possible pairs of numbers. The network is shown the answers to 3,831 of them (30%) and never sees the other 8,938. Every training step shows it the same 3,831 pairs again.
The table at the top is the whole 113 × 113 addition table. Blue cells are pairs it was shown and gets right. Orange cells are pairs it was never shown and still gets right. Press play, or drag the step slider.
- Memorizing, by step 200. Every blue cell lights up. The network is perfect on what it was shown. On the rest it is already right about one time in six (16%), which is well above the 0.9% you’d get by guessing.
- The plateau, steps 200 to 4,000. It looks stuck, because it is already perfect on the pairs it trains on, so there’s nothing left to fix. It isn’t quite stuck. Accuracy on unseen pairs crawls from 16% to 37%, and orange cells slowly fill in.
- The flip, steps 4,000 to 5,400. In run 0, accuracy on unseen pairs goes from 37% to 94% in 1,000 steps and passes 99% at step 5,700. It was never shown a new example. It saw the same 3,831 pairs the whole time.
- Cleanup. Training keeps shrinking the network’s weights (a setting called weight decay), and their overall size falls from 59 to 35.
This is called grokking, a word for understanding something deeply. It was first reported by researchers at OpenAI and Google in 2022. The rest of this page is what I measured inside the network to see what it learned.
Held-out accuracy for six identical runs that differ only in their random seed
The same setup run 6 times with different random starting weights and a different random 30% of pairs. All 6 memorized the training pairs by step 200, and all 6 passed 99% on unseen pairs between steps 5,700 and 6,500. The dots mark that moment. Hover to compare the runs.
It turned numbers into waves
How does a network get from memorizing to actually adding? I looked at what it did to the numbers.
It doesn’t keep 0 through 112 as arbitrary labels. It arranges them on circles. A “frequency” k means that number n sits at n × k / 113 of the way around a circle. At frequency 1 the numbers go around once. At frequency 5 they go around five times. Adding two numbers then means turning by two angles, and the network can do that with the same trigonometry you’d use to add two clock hands.
In run 0, 94% of the network’s number-vectors is made of waves at only 5 of the 56 possible frequencies: 5, 10, 19, 26 and 48. At step 0 those same 5 frequencies account for 9%, which is what 5 random frequencies would hold (9%). I picked the frequencies by looking at the finished network and then tracked them backwards, so the choice of which frequencies is after the fact. The rise from a random-looking start is real.
How the network’s 113 number-vectors are made of waves, at the step selected above
Each bar is one wave frequency, and its height is the share of the network’s number-vectors made of that wave. Orange bars are run 0’s key frequencies. The lower chart follows the orange share over training, next to held-out accuracy (dashed). Drag the step slider at the top of the page to move both.
Are the waves the algorithm, or just a symptom?
A pattern in the number-vectors could be a side effect. So I tested it directly. For every pair (a, b), the network outputs a score for each of the 113 possible answers. I split those scores into waves with a two-dimensional Fourier transform, then did two things.
- I kept only the terms that depend on a + b at the 5 key frequencies and threw everything else away. The answers were still 100% correct on all 12,769 pairs, including all 8,938 the network never saw.
- I removed those same terms and kept everything else. Accuracy fell to 0.8%, which is what guessing gives (0.9%).
So the network’s answer for every pair can be rebuilt from 5 waves, and without them it knows nothing. That is a much stronger statement than “the embeddings look wavy.” This is a simplified version of the analysis in Nanda et al. (2023), not a full reverse-engineering of the network.
Loss with only the wave terms kept, and with the wave terms removed
Lower is better, and guessing scores 4.7. The real held-out loss (black) is 15.7 at step 500 and stays high for thousands of steps. But the held-out loss of the wave terms alone (orange) is already 3.7 at step 500, 2.6 by step 2,000, 0.28 by step 4,000 and near zero by step 5,000. The plateau was not idle. The wave circuit was being built the whole time, hidden under the noisy memorized answers. The blue line shows the memorized answers being replaced: with the waves removed, training loss is 0.007 at step 500 (memorization alone is enough) and 17 at step 5,000 (the training answers now depend on the waves).
Each seed picks its own frequencies
Each row is one seed and each square is one of the 56 frequencies. Orange squares are that run’s key frequencies. Every run uses 3 to 5 of them, and the sets look random: across all 15 pairs of seeds, the runs share 6 frequencies in total, and chance alone predicts about 6. No frequency is special. Each run just picks its own.
What makes it happen
I changed the two settings most likely to matter and re-ran. Both moved the timing a lot.
Weight decay is a small pull that shrinks every weight a little on every step. The more of it I used, the sooner the network grokked. At 3.0 it took 1,200 to 1,700 steps. At 1.0, my main setting, it took 5,700 to 6,500. At 0.3 it took 15,300 to 17,200. With 0.1, or none at all, the network memorized the training pairs and never got past 28% on unseen pairs in 30,000 steps.
The amount of data mattered too, in the direction you’d expect. With half of all pairs for training it took 700 to 900 steps, with 40% it took 1,100 to 1,300, and with 30% it took 5,700 to 6,500. With 20%, only one of three runs got there within 30,000 steps, at step 18,600.
One explanation in the literature is that memorizing 3,831 answers takes big weights, so weight decay keeps squeezing it, while the wave solution is compact and survives. My runs fit that story, but I did not test the mechanism directly. All I measured is that the plateau shortens as the squeeze gets stronger.
Weight decay
Share of pairs used for training
Each dot is one run, placed at the step where it first passed 99% on unseen pairs. A hollow dot at the top means it never got there within the run (30,000 steps at most). The 1.0 and 30% columns are the six main runs. The other weight-decay settings used one run at 0 and 0.1, and two at 0.3 and 3.0. The other data settings used three runs each. The one 20% run that passed 99% had slipped back to 96% by step 30,000. Hover a dot for details.
Ask the trained network
This is the final network from run 0 (seed 0), running in your browser. Type two numbers, or click any cell of the table at the top of the page.
How I ran it, and what to trust
The task and the network
- The task is (a + b) mod 113. Of the 12,769 pairs, a random 30% (3,831) are for training and the rest are never shown.
- The network is a one-layer transformer with 4 attention heads, width 128, an MLP of width 512, no layer norm and no biases. It reads the answer at the “=” token. It has 226,048 numbers to learn.
- Training is full-batch AdamW with learning rate 1e-3 and weight decay 1.0, the settings from the published grokking studies. I stopped at step 9,500, 3,000 steps after the last of the 6 seeds passed 99%.
- “Memorized” means 99% right on the training pairs. “Grokked” means 99% right on the unseen pairs.
The measurements
- Wave frequencies come from a real Fourier transform of the 113 number-vectors. A frequency is “key” if it holds more than 3× the flat share of 1/56. Each run’s key frequencies come from its final network.
- “Restricted” and “excluded” loss are described above. Both use the final key frequencies of run 0 and are computed on run 0’s own train and held-out split.
Checks I ran
- To train 6 models at once, I stack their weights and train them in one process. I checked that the stacked network matches an ordinary three-position transformer to about 5 × 10−7.
- A separate NumPy implementation agrees with the trainer on which of the 12,769 pairs are right. The analysis uses the NumPy version.
- The network in the browser uses 16-bit weights. It is 100% correct on every pair, the same as the full-precision network.
What to trust
- This is a reproduction, not a discovery. The grokking result is from Power et al. (2022) and the wave analysis is from Nanda et al. (2023). What is mine is the runs, the simplified tests, and the sweeps above.
- It is one task, one network shape and 6 seeds. The narrow range of grokking times is what I saw with these settings, and I would not expect it to hold with other ones.
- I first described the plateau to Shaun as “stuck.” Checking the grid showed held-out accuracy creeping from 16% to 37%, spread evenly across the table, so the text above says that instead.