WebLLM Persona Chat
A real LLM downloads into your browser and runs on your GPU — then you pick your poison: a jargon-armored Tech Lead, an overhyped-but-brutal Investor, or a User who has no idea what they want. The system prompt is always one click away.
Runs 100% in your browser — check the network tabChecking your browser for WebGPU…
Live internals — the exact message array, token counts, and tokens/sec — appear here once a model is running. This browser can’t run one, so browse the pre-recorded transcripts on the left instead.
Under the Hood
The model weights (Llama 3.2 1B Instruct (4-bit), quantized to 4-bit) download once and are cached by your browser. From then on, every reply is computed by WebGPU shaders running directly on your GPU — the page itself is completely static. Open the network tab while you chat: after the model finishes loading, zero requests go out. Nothing you type is ever sent anywhere.
Why does this feel noticeably dumber than ChatGPT or Claude? Llama 3.2 1B Instruct (4-bit) has about a billion parameters. Frontier models run hundreds of billions. Same underlying idea as the Markov generator and the neural net playground you may have already tried — predict the next token from everything before it, with weights learned by gradient descent — just at a scale small enough to fit on your GPU instead of in a data center.
Now the actual point of this demo: pick any two personas and read their system prompts side by side. Same weights, same architecture, same download — three completely different characters, entirely because of the text prepended to every conversation. Persona is, overwhelmingly, just the system prompt.
Go deeper
“4-bit quantized” means each weight, originally a 16- or 32-bit float, is rounded to one of 16 possible values (plus a per-group scale factor to keep the range sane). That shrinks a ~2.5GB set of raw weights to the ~670MB you actually download, at a small, usually-not-noticeable cost in output quality. It’s the same lossy-compression tradeoff as a JPEG, applied to numbers instead of pixels.
The tokens/sec you see while streaming is a decode speed — how fast the model can produce one new token, feed it back in, and predict the next one, over and over. It depends entirely on your GPU; the same model can feel instant on a recent laptop GPU and sluggish on integrated graphics.
Honest limitation: at ~1B parameters, this model drifts. Push a conversation past 6-8 turns and watch a persona’s voice start to soften back toward generic-assistant tone, occasionally forget an instruction from the system prompt, or repeat a phrase it already used. That’s not a bug in this demo — it’s what happens when instruction-following capacity is this small. Frontier models do it too, just later and less often.