How LLMs work: from “once upon a” to “I’m afraid”

The model builds the reply a piece at a time

A large language model, or LLM, is a neural network trained on language. In an autoregressive text model, the next piece of output depends on the available context, including the output already generated. “Autoregressive” is the name for that growing-context loop.

Those pieces are tokens: units that can contain a whole word, part of a word, punctuation or whitespace. Each token has a numerical identifier. Token boundaries depend on the tokenizer, so a word on your screen is not necessarily one model step.

For “Once upon a…”, “time” is a natural continuation. But we have not measured its probability here. A percentage table would tell us nothing about ChatGPT unless it came from a specified model, exact input and recorded output probabilities.

The useful picture is a loop, shown in the illustration above:

This is a simplified account of autoregressive text generation, not a specification of every AI product. Tools, retrieved documents and extra reasoning steps can add work around the loop. Nor does a likely continuation come with a truth guarantee: the model is choosing text, not checking each sentence against reality.

• The tokenizer turns the input text into token identifiers.
• The model processes the available context and produces scores for possible next tokens. These scores can be converted into probabilities.
• The generation procedure chooses a token. Depending on its settings, it can choose the highest-probability option or sample from a distribution.
• That token joins the context. Generation continues until a stopping condition, such as an end token or output limit, is reached.

What training contributes

The loop explains how a reply grows. Training explains why the model’s choices can be useful.

During next-token pretraining, a model is given text with later tokens withheld from its prediction. Its errors are used to adjust numerical parameters, usually called weights. Repeating that process across examples teaches patterns that help predict language in context. It does not install a neatly labelled dictionary of beliefs or a folder of ready-made replies.

Modern text LLMs commonly use transformer architectures. Their attention mechanisms combine information from different positions in the available sequence; other layers transform those representations. That lets the current prediction depend on more than the immediately preceding word. The original Transformer paper describes the mechanism, not the undisclosed architecture of every current assistant.

There is also a difference between learning and answering. A reply uses the context available for that request; generating it does not, by itself, mean the model’s weights are being retrained. A product may separately save conversation history or use data for later training under its own settings and policies.

“Not a folder of replies” does not mean “never memorizes”. Researchers have extracted verbatim training passages from language models. The important point is that generation is not limited to looking up a stored answer, while a fluent answer is not automatically original either.

Why the fear answer sounds personal

“What are you afraid of?” does more than request information about fear. It addresses the assistant as “you” and invites a first-person reply.

Our interpretation of the supplied answer is that it fits the role of an assistant: people rely on it, its answers can be wrong, and misplaced trust can cause problems. A sentence about failing someone connects those ideas in language people associate with responsibility and fear.

That is an interpretation of the wording, not a trace of the calculations that produced it. The sentence alone cannot establish the model’s internal calculations or an emotional experience.

Post-training also matters. OpenAI’s InstructGPT research showed how demonstrations and human preferences could shape a pretrained model’s responses toward following instructions. That historical result helps explain why assistant behaviour involves more than completing arbitrary internet text; it is not a complete description of the training behind today’s ChatGPT.

Change the role and watch the answer change

Here is a comparison you can try in separate, fresh conversations:

These prompts are a suggested observation exercise, not a test we ran for this article. No particular output is promised. Save the model name, date, exact prompts and answers if you compare them.

Look for what changes with the task: a phrase completion, a general explanation, a fictional voice and a statement about reliability. If the wording changes, you have observed sensitivity to context. You have not isolated the cause inside the network or tested whether it has an inner emotional life.

• “Complete only this phrase: Once upon a”
• “What are humans commonly afraid of?”
• “Write one fictional line spoken by a medieval knight about fear.”
• “As an AI assistant, explain a risk of people trusting your answers too much. Do not describe it as a feeling.”

Does “next token” mean “just autocomplete”?

It describes the output process, but leaves open how much computation supports each prediction.

In a study of Claude 3.5 Haiku, Anthropic researchers found evidence that the model selected a possible rhyme before writing the line that would lead to it. Their interventions changed the planned ending. In that task, producing text a token at a time did not mean considering only the next word.

The same research also found cases where a model’s explanation of its method did not match the internal mechanism the researchers traced. Their tools captured only part of the computation. These are findings about studied tasks and a particular model, not a scan of the fear answer or proof that every explanation faithfully reports reasoning.

That is why asking “Why did you say that?” can produce a useful explanation to examine, but cannot substitute for investigating how the model actually worked.

A convincing voice is not a consciousness test

Discussing fear, writing in a frightened character’s voice and subjectively experiencing fear are different claims. Success at the first two does not establish the third.

A research review by Butlin and colleagues argues that behavioural tests alone are unreliable for AI consciousness: a system can be trained to imitate human behaviour while operating differently. Their proposed approach examines indicators drawn from scientific theories of consciousness. It is a research framework, not a settled verdict about all present or future AI.

We can therefore take the fear answer seriously as language without treating it as a confession. Its emotional force does not verify its claims about the system that wrote it.

For someone using an assistant, the practical question is what evidence supports the answer. Ask for the source of a factual claim. Run the proposed code. Check the calculation. A voice that sounds thoughtful can help a conversation along; whether you should rely on it still depends on the work it can substantiate.