Tiny LM
I trained a small language model from scratch on seven old books about business and self-help. It runs in your browser with no machine learning library. Type a sentence and watch it cut the text into tokens and guess the next word.
- Parameters
- 3,737,088
- Parity with PyTorch
- within 1.2e-5, on five fixed inputs
- Download
- 14.9 MB, fetched once
- Training
- 4,500 steps, 7 minutes, Apple M4 Pro GPU
What it showed
You can watch every step a language model takes, with nothing hidden inside a library. A model this small picks up the sound of its books, and its sentences wander after a few words.