Language & Large Language Models
From word counts to transformers that write, code and reason.
A chatbot writes its reply one small piece at a time, and every piece comes from the same deceptively simple skill: predicting what text comes next. This lesson follows that idea from counting words to transformers, and is honest about what it does and does not explain.
What you’ll learn
- Explain what a language model predicts and how text is generated from it.
- Describe how tokenization and embeddings turn text into vectors.
- Explain what attention does and why a causal mask is needed.
- Describe how sampling settings shape output, and how pretraining, fine-tuning and RLHF turn a predictor into an assistant.
Predict the next word
Finish this: "I'd like a cup of …". You probably thought "coffee" or "tea", and certainly not "gravel". You just ran a language model in your head: a probability for every possible next word, given the words so far.
Formally, a language model gives . To generate, sample one token from that distribution, append it, and repeat. Everything a chatbot writes is produced this way, one token at a time.
The oldest way to get the probabilities is to count, an idea Claude Shannon was already playing with in 1948. An n-gram model, a workhorse of early natural language processing, looks only at the last words:
In a corpus, "the cat" appears 200 times, and 30 of those are followed by "sat". What does the trigram model give for , in percent?
Show the answer
15 %
, so 15%. The catch: most longer phrases never occur in any corpus at all, so their count, and their probability, is zero. Modern models replace counting with a neural network that generalizes from similar contexts.
Text becomes tokens
What does a model actually read? Not letters, and not quite words. Tokenization splits text into tokens from a fixed vocabulary, typically tens of thousands to a few hundred thousand entries as of the mid-2020s. Common words are single tokens; rare words are split into familiar pieces.
The usual recipe is byte-pair encoding: start from single characters (or bytes) and repeatedly merge the most frequent adjacent pair into a new token. No word is ever "unknown": at worst it is spelled out in small pieces.
Words as vectors
Each token id is looked up in a table of embeddings: a vector of hundreds to thousands of numbers. The vectors are learned, so that tokens used in similar contexts end up pointing in similar directions. "You shall know a word by the company it keeps," as the linguist J. R. Firth put it in 1957.
Closeness is measured by cosine similarity. Directions can even carry relations: word2vec (Mikolov et al., 2013) became famous for .
Think first Nobody told the model that "kitten" and "cat" are related. How do their vectors end up close?
They appear in similar contexts ("the ___ purred", "fed the ___"), and training rewards vectors that help predict those contexts. Similar usage produces similar vectors. The same mechanism also absorbs the text's stereotypes: embeddings trained on web text have been shown to encode gender biases about occupations.
Attention: context decides meaning
In "The animal didn't cross the street because it was too tired", what is "it"? The animal. Change "tired" to "wide" and "it" becomes the street. A word's meaning depends on its context, so a fixed vector per word is not enough.
Attention fixes this. Each token forms a query, compares it with every token's key, turns the match scores into weights with a softmax, and takes the weighted average of their values. The result is a new, context-aware vector for that token. Because every token attends to the same sequence, this is self-attention, the heart of the transformer (Vaswani et al., 2017):
A transformer is trained to predict every next token of a sentence in one parallel pass. Why does it need the causal mask?
- To make the attention weights add up to one
- To speed up training by skipping half the computation
- To remove common words like "the" from attention
- So no position can peek at the later tokens it is supposed to predict
Show the answer
D. So no position can peek at the later tokens it is supposed to predict
Without the mask, position could just attend to token and copy the answer. The mask hides the future, so each prediction uses only earlier text, exactly as at generation time.
Choosing the next token
After its final layer, the model outputs a probability for every token in its vocabulary. How you pick from that distribution changes the text. Always taking the top token (greedy decoding) is predictable but can be bland or repetitive. Temperature reshapes the odds: below 1 sharpens them, above 1 flattens them. Top-k and top-p sampling cut off the long tail of unlikely tokens.
The next-token probabilities are . Top-p sampling with keeps the smallest set of top tokens whose probabilities add up to at least . How many tokens does it keep?
Show the answer
3 tokens
is not enough; adding gives . So 3 tokens survive, renormalized to sum to one, and the tail is never sampled.
From predictor to assistant
Pretraining runs next-token prediction over a huge corpus (trillions of tokens of web text, books and code, as of the mid-2020s). To predict well, a model must absorb grammar, facts, styles and patterns of argument. Scaling laws (Kaplan et al., 2020; Hoffmann et al., 2022) found that loss falls smoothly, as a power law, with more parameters, data and compute. And GPT-3 (2020) showed that a big enough model can pick up a task from a few examples in its prompt: in-context learning.
But a pretrained model only continues text: ask it a question and it may reply with more questions. Two more stages turn an LLM into an assistant:
| Stage | Training signal | What it adds |
|---|---|---|
| Pretraining | The next token of raw text | Broad knowledge and fluency |
| Fine-tuning | Example conversations written by people | Following instructions in an assistant format |
| RLHF | People's comparisons of model responses | A reward model learns their preferences; the model is tuned toward them |
In the Hidden Layer you'll build each layer of this stack: From N-grams to Embeddings (classic language modeling and tokenization), Attention & Transformers (queries, keys, values and the full architecture), Large Language Models: Pretraining & Scaling, and Fine-Tuning, RLHF & Prompting, which turns a text predictor into a helpful assistant.
Key takeaways
- A language model predicts a probability for every possible next token; generation repeats predict, sample, append.
- Text is split into subword tokens (for example by BPE), and each token becomes a learned vector in which related words lie close together.
- Attention rebuilds each token's vector from a weighted mix of the others; the causal mask keeps next-token training honest.
- Temperature, top-k and top-p trade predictability for variety; none of them makes the output true.
- Pretraining on huge corpora gives broad abilities; fine-tuning and RLHF turn the predictor into an assistant.
Check yourself
In the interactive lesson these are scored with your confidence, and what you miss comes back for review.
What is a large language model trained to do during pretraining?
- Look up answers in a stored database of facts
- Predict the next token of text, over and over, across a huge corpus
- Follow human instructions, from the very start
- Translate sentences into formal logic and reason over them
Show the answer
B. Predict the next token of text, over and over, across a huge corpus
Pretraining is self-supervised next-token prediction: the text supplies its own labels. Instruction following comes later, from fine-tuning and RLHF.
You lower the sampling temperature from 1.0 to 0.2. What happens to the next-token distribution?
- It flattens, so unlikely tokens are picked more often
- The model learns from its mistakes and becomes more accurate
- It sharpens, piling probability onto the most likely tokens
- Nothing: temperature only changes generation speed
Show the answer
C. It sharpens, piling probability onto the most likely tokens
Dividing the scores by exaggerates their differences, so the softmax concentrates on the top tokens; as sampling becomes greedy. The output gets more predictable, not more correct.
In "The trophy didn't fit in the suitcase because it was too big", what lets a transformer link "it" to "trophy"?
- Self-attention: the vector for "it" is rebuilt from a weighted mix of other tokens, and training can give "trophy" a high weight
- A hand-written grammar rule that a pronoun refers to the first noun
- The causal mask, which joins each pronoun to the nearest noun
- Tokenization, which merges "it" and "trophy" into one token
Show the answer
A. Self-attention: the vector for "it" is rebuilt from a weighted mix of other tokens, and training can give "trophy" a high weight
Attention lets each position pull information from other positions, with weights computed from query–key matches. Nobody writes the rule: training on lots of text shapes the projections so that useful links get high weight.
In RLHF, what do the human raters mainly provide?
- The full text of every answer the model will ever give
- Hand-edited weights for the network
- A list of true facts for the model to memorize
- Comparisons of model responses, used to train a reward model that the language model is then tuned to score well on
Show the answer
D. Comparisons of model responses, used to train a reward model that the language model is then tuned to score well on
Raters compare candidate responses; a reward model learns to predict their preferences; reinforcement learning then shifts the language model toward responses the reward model scores highly (Ouyang et al., 2022).
Go deeper
- From N-grams to Embeddings · Classic language modeling and tokenization.
Further reading
- D. Jurafsky & J. H. Martin, Speech and Language Processing (3rd ed. draft), chapters on n-grams, embeddings, transformers and LLMs, free at web.stanford.edu/~jurafsky/slp3.
- A. Vaswani et al., "Attention Is All You Need" (NeurIPS, 2017).