Where does an LLM’s knowledge come from—and what changes after pretraining? Follow one example: a child has two apples and buys three more. Then add a request: “Show the equation before giving the answer.”
Pretraining builds foundations
Training on large collections of text and code can teach word meanings, factual associations, language patterns and some problem-solving abilities. For autoregressive models, predicting subsequent text is a common training exercise. Knowledge and capabilities are results of that learning, expressed through parameters rather than a literal document store.
Post-training continues task learning
Getting five is only part of this request; the answer should also show 2 + 3 = 5. Further training can shape instruction-following and improve task abilities.
- SFT: train on demonstrations of good responses.
- DPO: train from preference pairs. Both apple answers may be numerically correct, while one better follows the requested format. The objective concerns relative preference, not a guarantee that every chosen answer’s absolute probability increases.
- Verifiable rewards: sample attempts during training and check answers or executable tests to supply reward.
- Classic RLHF: human comparisons train a reward model; its scores then guide reinforcement learning of the answering model.
These are not mutually exclusive stages or a universal fixed recipe. Both pretraining and post-training can improve capabilities; neither guarantees correct answers on unseen problems.
Animation and scope
The accompanying 3m33s animation has Chinese captions, music and no narration. Answers, parameter grids and training updates are teaching illustrations, not measured LLM training results.