What is a Large Language Model?

A Large Language Model (LLM) is trained to do exactly one thing: given a sequence of text, predict what comes next. That's it. Everything else — chat, code, reasoning, tool use — emerges from doing this well enough at massive scale. When you send Claude a message and get a response, under the hood it's producing one token at a time, each conditioned on everything before it.

The model doesn't see words — it sees tokens, which are pieces of text produced by a tokenizer. Common words are usually one token; rare words split into several; punctuation and whitespace are their own tokens. Roughly: 1 token ≈ 4 English characters, or about ¾ of a word. This matters because you pay per token (input + output) and because context limits are measured in tokens.

'Hello world' → 2 tokens
'antidisestablishmentarianism' → 6 tokens
'Vikram Grover' → 4 tokens

Rule of thumb: 1000 tokens ≈ 750 English words

Training is the (very expensive, weeks-long) process of adjusting model weights on massive text corpora. Inference is when you use the trained model — sending a prompt and getting a response. You interact with the trained model; you don't retrain it. Post-training techniques like RLHF (reinforcement learning from human feedback) are what turn a raw language model into a helpful assistant like Claude.

Context window = the maximum number of tokens the model can consider at once (both your input and its output). Claude models today typically have 200K token windows. Sounds huge, but it fills fast: a 100-page PDF is ~50K tokens, and long conversations accumulate. Everything relevant to the current turn — system prompt, history, tool results, retrieved documents — must fit in this budget.

At each step, the model outputs a probability distribution over the next token. Sampling picks one. Temperature controls how peaked or flat that distribution is. Temperature 0 = always pick the most likely token (nearly deterministic). Temperature 1 = sample from the raw distribution (more creative, more varied). For structured tasks, lower is usually better; for creative writing, higher.

The model is a pattern-completion engine, not a truth engine. When asked something it doesn't know, its training pushes it toward producing text that LOOKS plausible — which sometimes means confidently inventing facts. This is called hallucination. Grounding the model in retrieved documents (RAG) and asking it to cite sources are the standard mitigations.