Skip to main content
Concept

What are model parameters?

The billions of numbers a model learns during training, and what a label like "7B" or "70B" is really telling you.

The short version

  • → Parameters are the adjustable values inside a model that hold what it learned.
  • → A label like 7B or 70B means 7 billion or 70 billion parameters.
  • → More parameters usually means more capability but also more cost and slower, heavier serving.

A model's parameters are the numbers inside it that get tuned during training. They are where everything the model "knows" is stored. A modern language model has billions of them, which is why you see names like Llama 7B or a 70B model.

The count is a rough measure of a model's capacity: how much pattern and nuance it can hold. It is the single number people reach for when sizing a model, though it is far from the whole story.

What a parameter actually is

Think of each parameter as a tiny dial. Training turns billions of these dials, a little at a time, until the model predicts text well. The final settings, taken together, are the model's weights. There is no separate fact store: the knowledge is spread across all of them.

What "7B" and "70B" mean

The B is billions of parameters. A 7B model has about seven billion dials; a 70B model has ten times more. Bigger models generally capture more nuance and handle harder tasks, but they also need much more memory and compute to run. Some newer models list two counts, such as 120B total and 5B active. That is a mixture-of-experts design, where only part of the model works on each token.

Why bigger is not automatically better

Beyond raw size, quality of training data, the training recipe, and fine-tuning matter enormously. A well-trained smaller model can beat a poorly trained larger one on real tasks. And a huge model is slower and costlier to serve, which is often the wrong tradeoff for everyday use.

Size shapes cost and speed

Parameter count drives how much hardware a model needs and how fast it responds. Small models can run on a single GPU or even a laptop; the largest need clusters. This is why providers offer a range: a small fast model for routine work and a large one for the hardest problems.

An analogy

Imagine a mixing board with billions of sliders. Training is the long process of setting every slider so the output sounds right. The parameter count is just how many sliders the board has. More sliders allow finer control, but only if they were set well.

Go deeper

How a neural network works

By the end you will have computed a small neural network by hand and know exactly what a large one is doing more of. 17 min read →

Where Berges AI fits

Berges AI does not default you to the biggest model for everything. You pick from the sidebar: DeepSeek is the lean, fast one, and GLM, Qwen and Kimi are larger ones for harder problems. The tradeoff is yours to make, turn by turn.

Related concepts

What is a mixture of experts?

How a model can have hundreds of billions of parameters and still answer quickly: it only wakes up a few of them for each token. Read →

How transformers work

The neural network design that powers almost every modern language model, and the one idea, attention, that made it work. Read →

Open weights vs closed models

Whether a model's trained parameters are public or locked behind an API changes who controls it, where it runs, and what you can do with it. Read →

Inference and hosting

Training builds the model once. Inference is every time you actually use it, and hosting is the infrastructure that keeps it fast. Read →

Things people ask

What does 70B mean in AI?

It is short for 70 billion parameters, the learned values inside the model. It is a size label, not a version number. A 70B model is roughly ten times the size of a 7B one and needs roughly ten times the memory to run.

What does the B in a model name mean?

Billion. 7B, 70B, and 405B are parameter counts. Some models list two numbers, total and active, because mixture-of-experts models only use part of their parameters on each token.

Does more parameters mean a smarter model?

Usually more capable, but not always smarter in practice. Training quality and tuning matter just as much, and a well-built smaller model can outperform a larger, weaker one.

What is the difference between parameters and weights?

They are nearly the same thing. Parameters are the tunable values in the model; weights are those values after training. People use the terms almost interchangeably.

Why not just always use the biggest model?

Big models are slower and far more expensive to run. For most tasks a smaller model answers just as well, much faster and cheaper, which is why providers keep a range on offer.