TinyStories ARM
Explore

Explorer: one prompt through the real model

Every number below is the real model's: Large 124M best design · stage 2 · FineWeb-Edu full + Cosmopedia · 2.5B tokens · context 512, the project's base model. A prompt's tokens become vectors, go through every transformer block (attention mixes the tokens, the feed-forward layer refines each one) and come out as a probability for every token in the vocabulary. Hover a row to follow a token, click a part for its explanation, press ▶ Play the data flow to watch one token go through a block stage by stage, or switch to Diagram for the queries, keys and values. Recorded on an Apple M5 (2026-11-18 07:03); the app explores any prompt live.

Block 1 Head
Attention

Logit lens: what the model would predict after each block

The final norm and output layer applied after every block, as if the model stopped there. Each cell is the top guess for the token after that column's token (hover for the top 3). Watch the guesses sharpen as the blocks add up.