Talk to the model, in your browser
The project's 124M model runs right here: wllama (llama.cpp compiled to WebAssembly, on the GPU through WebGPU when your browser has it) downloads the 8-bit copy from Hugging Face once (174 MB, then cached) and generates on your own machine. Nothing you type leaves the page. It's a 124M model trained on 2.5B tokens: it writes fluently and is often wrong on facts, and it repeats itself on long answers.
How this works
- The model was exported exactly to the standard Qwen3 format and converted to GGUF (plan steps S1 and S7: the same scores as the original on every benchmark). The files are public: chat, base.
- The chat model writes in a plain
User: …/Assistant: …format; the GGUF carries that chat template, so this page sends messages the same way an OpenAI-style client would. - On the GPU, all 12 blocks and the output layer run as WebGPU shaders (Metal on a Mac, Vulkan or Direct3D elsewhere); only the token embedding table (50,257 × 768, 39 MB) stays on the CPU, since each step reads just one row of it. On the CPU the WebAssembly build runs on one thread: several threads need the page to be cross-origin isolated, which GitHub Pages can't switch on.
- To run it on your own computer instead:
ollama run hf.co/aramkris/fineweb-edu-124m-chat-dpo-qwen3:Q8_0, orllama-server -hf aramkris/fineweb-edu-124m-chat-dpo-qwen3:Q8_0 --jinja -c 512.