What is fine-tuning?
Taking a capable general model and training it a little more on your own examples to make it better at one specific job.
The short version
- → Fine-tuning continues training a pretrained model on a smaller, focused dataset.
- → It changes the model's behavior and style, not the facts it can look up.
- → For adding knowledge, prompting or retrieval is usually cheaper and easier than fine-tuning.
Pretraining builds a broadly capable model at great expense. Fine-tuning takes that finished model and trains it a bit further on a smaller set of examples, to specialize it for a particular task, tone, or format.
It is the difference between hiring a smart generalist and giving them a week of on-the-job training for your specific role. The raw ability was already there; fine-tuning points it in a direction.
How it works
You collect examples of the behavior you want, often input and ideal-output pairs, and continue the training process on just those. The parameters shift to favor that pattern. Because you start from an already capable model, it takes far less data and compute than training from scratch.
What it is good at
Fine-tuning excels at shaping form and behavior: matching a house style, always replying in a fixed structure, handling a narrow domain's phrasing, or being consistent at a specialized task. If you need the model to reliably act a certain way, fine-tuning is a strong tool.
What it is bad at
It is a poor way to add facts. Fine-tuning teaches patterns, not a reliable, updatable knowledge base, and baked-in facts go stale the moment they change. For "know this information", retrieval that fetches the facts at question time is usually better, cheaper, and easier to keep current.
The cheaper alternatives to try first
Before fine-tuning, most needs are met by a good prompt with clear instructions and a few examples, or by retrieval for knowledge. These need no training run and are trivial to change. Reach for fine-tuning when prompting has genuinely hit its ceiling and you need consistency at scale.
An analogy
A pretrained model is a talented new hire. A prompt is the instructions you give them for one task. Fine-tuning is putting them through a training course so a whole class of tasks becomes second nature. Retrieval is handing them the reference manual to look things up.
Go deeper
How post-training shapes a model
By the end you will know how a text predictor is turned into an assistant, and why the confidence, warmth, and agreement it shows were selected by raters rather than earned by being right. 17 min read →
Where Berges AI fits
Berges AI shapes behavior without fine-tuning. Its pipelines and house rules adjust each response at use time, asking for no filler or for step-by-step working, so you get a tuned feel with none of the cost, delay, or staleness of retraining a model.
Related concepts
How AI models are trained
The process that turns a random network into something useful: read a lot, adjust, repeat, then teach it to behave. Read →
Open weights vs closed models
Whether a model's trained parameters are public or locked behind an API changes who controls it, where it runs, and what you can do with it. Read →
What is RAG?
Letting a model look things up. Instead of answering from frozen memory, it fetches the relevant text first, then answers from that. Read →
Things people ask
When should I fine-tune instead of just prompting?
When you need consistent behavior at scale and a well-crafted prompt no longer gets you there. For most tasks, and especially for adding knowledge, prompting or retrieval is enough and much simpler.
Can fine-tuning teach a model new facts?
It can, but poorly. The facts get baked in, are hard to update, and can still be recalled unreliably. For knowledge, retrieval that supplies the facts at question time is the better approach.
Do I need open weights to fine-tune?
Not necessarily. Some closed providers offer managed fine-tuning through their API. Open-weight models give you the most freedom to fine-tune however you like on your own infrastructure.