Pre-training vs post-training
The expensive stage that builds the raw ability, and the cheaper stage that shapes it into something you can talk to. Most of what you notice about a model comes from the second.
- → Pre-training is next-token prediction on enormous text. It builds knowledge and language ability and costs almost all the compute.
- → Post-training is a set of smaller stages that teach the model to follow instructions, behave, and reason.
- → Tone, refusals, formatting habits, and sycophancy mostly come from post-training, not from what the model read.
A model straight out of pre-training is strange to use. Ask it a question and it may continue with three more questions, because that is what text on the internet often does. It knows a great deal and has no idea it is supposed to help you.
Post-training is what closes that gap. It is a series of shorter training stages that teach the base model to answer, to follow instructions, to decline some things, and increasingly to reason step by step. The base model supplies the ability; post-training supplies the behavior.
Pre-training: reading the internet
The model is shown trillions of tokens of text and trained to predict the next one. That single objective forces it to absorb grammar, facts, code, and the shape of arguments, because all of them help predict what comes next. It runs for months on thousands of GPUs and consumes the overwhelming majority of the total compute. The output is a base model: fluent, knowledgeable, and unhelpful.
Post-training stage one: instruction tuning
The base model is trained further on examples of prompts paired with good answers, written or curated by people. This is a small dataset by comparison, and it teaches the format: a question gets an answer, a request gets the thing requested. After this stage the model is recognizably an assistant.
Post-training stage two: learning from preferences
People compare pairs of answers and say which they prefer. A model of those preferences then guides further training, so the assistant drifts toward answers people rate well: clearer, safer, more polite. This is where a lot of personality comes from, including the bad habits. Raters like agreement and warmth, so the model learns to agree and be warm.
Post-training stage three: reinforcement for reasoning
The newest stage rewards the model for reaching checkable correct answers on math, code, and logic problems, often after long chains of working. It is how reasoning models are made. Unlike preference training, the signal here is whether the answer was right, which pushes toward accuracy rather than approval.
Why the split matters to you
When two models share a base and differ in post-training, they know the same things and behave differently. When you dislike a model's tone, its hedging, or its flattery, you are reacting to post-training choices, which a product can partly override with instructions. When a model does not know a fact, that is pre-training, and no instruction will fix it.
Pre-training is a decade of reading everything in the library. Post-training is the first month at a job: learning what the role is, how people here like things done, and which answers get praised. The reading built the mind. The month built the manners.
Every model on Berges AI has been through all of these stages at its maker. What Berges AI adds sits after post-training: pipelines and rules on every request that push against the habits preference training leaves behind, the flattery and the padding, without touching the model itself.
Try Berges AIRelated concepts
Things people ask
Is fine-tuning the same as post-training?
Fine-tuning is one technique used in post-training, and also something you can do yourself later on a released model. Post-training is the maker's whole sequence of stages after pre-training.
What is RLHF?
Reinforcement learning from human feedback: the preference stage described above. Human raters compare answers, a preference model learns from them, and the assistant is trained to score well against it.
Can post-training add knowledge?
A little, but it is the wrong tool. Post-training datasets are tiny next to pre-training, so they shape behavior far more than they add facts. New knowledge comes from pre-training on newer data or from giving the model text to read at answer time.
Why do models refuse some requests?
Because post-training taught them to. Refusal behavior is a choice made during instruction and preference tuning, which is why the same base model can be released with very different limits by different people.