How ChatGPT answers a question
The full path of a single prompt — tokenizer, embeddings, attention, the transformer stack, sampling and safety — narrated component by component.
You type a question
You type a question and press enter. What happens next takes a few hundred milliseconds and runs hundreds of billions of multiplications through a pile of numbers so large it needs hundreds of gigabytes just to sit still. This is the whole path, from your keyboard and back again.
API gateway
Your text travels over an encrypted connection to a gateway, one of thousands of front door servers. It checks your session, picks a data centre with a free graphics processor, and puts you in a queue behind everyone else.
Auth and rate limits
Before any thinking happens the system checks who you are and how much you have already used. Rate limits exist because one long answer can occupy a chip worth more than a family car for several seconds at a time.
Input safety check
A fast classifier reads your prompt first. It is a far smaller model whose only job is spotting requests the assistant should refuse, and it finishes in a couple of milliseconds, so you never notice it was there.
Tokenizer
Now your sentence is chopped into tokens, the model's real alphabet. Common words are a single token and rare ones split into pieces. A useful rule of thumb is that one hundred tokens is about seventy five English words.
Embedding table
Each token is looked up in a giant table and becomes a long list of numbers, often several thousand of them, called a vector. In that space, words with related meanings sit near each other. That is where meaning enters at all.
Attention
Attention is the whole trick. Every token looks at every other token and decides which ones matter to it. In the sentence about the animal that would not cross the street because it was too tired, attention is what ties the word it back to the animal.
Transformer stack
One attention step plus a small feed forward network makes a single layer, and large models stack dozens of them, sometimes more than a hundred. Lower layers handle grammar and spelling, upper layers handle meaning, tone and intent.
Next token scores
At the very top the model produces a score for every token in its vocabulary, often well over one hundred thousand of them. That is genuinely all it does. Guess the next chunk of text. Nothing else is happening in there.
Sampling
Those scores become probabilities and one token is drawn from them. Temperature controls the gamble. Set it to zero and the model always takes the likeliest word; raise it and the draw loosens, which is exactly why the same question can give you different answers.
Key value cache
Producing the second token would mean redoing all that work, so the model stores its intermediate attention values in a key value cache. Without it, a thousand word answer would cost hundreds of times more than it does.
Streaming reply
Tokens are pushed to your screen the instant they are produced, which is why answers arrive word by word rather than all at once. The model has no idea how its own sentence ends when it begins writing it.
Output safety check
The finished text is checked again on the way out, and the model itself was shaped during training by human feedback to prefer helpful and honest replies. Safety is never one filter; it is several layers doing different jobs.
Conversation history
Your question and the reply are stored so your next message can include them. The model keeps no memory of its own between calls, so every follow up quietly resends the entire conversation, which is why long chats cost more.
Logging and evaluation
Finally the request is timed, logged and sampled for evaluation. Millions of these traces reveal where the model is slow, wrong or unhelpful, and the worst of them become the training material for the next version.
Watch it explain itself
Every step above is narrated aloud. Play it, or open it in the editor and make it yours — no account needed.