A language model is training in this browser tab.
Your GPU is training a small GPT in this tab right now. I wrote the training loop as WebGPU shaders, so it runs in a normal browser and there's nothing to install.
It's learning to write Norwegian from FineWeb 2's Bokmål slice. It starts out writing complete garbage, after a few minutes it spells real words, and after a while it produces something that almost looks like language.
This session
Train loss
The model writes
FAQ
Is it actually training?
Yes. The whole training loop runs on your GPU, and the loss you see is read back from it every step. If you want to check, download the checkpoint twice a minute apart and compare the files. They'll differ because the model trained in between. The checkpoint is a standard safetensors file, so you can also load it in Python and inspect the weights.
What is training?
The model reads 128 tokens and tries to predict the next one at every position, and the loss measures how wrong those guesses were. Backpropagation works out how each of the 24 million weights should change to do a little better, and the optimizer applies the changes. That's the whole loop, and it runs several times a second while the tab is open.
Can it answer questions?
No. 24 million parameters is not enough to store facts, so everything it writes about people or places is made up. It only learns what the language looks like. Chatbots like ChatGPT are hundreds of times bigger and get an extra round of training to answer questions.
What model is it?
A GPT with 24 million parameters: 10 layers, a 16k vocabulary and 128 tokens of context. The architecture borrows from the nanoGPT speedrun (RMSNorm, rotary embeddings, tied weights), just at a much smaller size, which keeps training fast enough to watch on a normal laptop.
Why is it faster on some machines?
The page checks your GPU's memory and picks the largest batch size that fits. The model is the same on every machine, but better hardware trains more tokens per second.
Will it hurt my GPU or battery?
No. Games push a GPU much harder than this does. It will use some battery and the fans may spin up. Mine do. There's a pause button if the noise bothers you.
Can I contribute?
Yes. The engine is a fixed graph of hand-written WGSL kernels in the spirit of llm.c, and there's plenty left to improve: the matmuls run at about a teraflop, attention hasn't been optimized yet, and the training recipe has headroom. The repo is linked at the top.
Why Norwegian?
The text is openly available, I like the language, and no big lab cares about it.