Open weights vs. closed models: when the downloaded one is enough

The debate about open and closed models is conducted like a football derby, which is a pity, because it isn't a matter of faith. It's a decision made separately for each task — and for most of them it has a clear answer.

What the two terms mean

Open weights means that you download the model file and run it anywhere. It doesn't necessarily mean open source code or published training data — it means you have the model itself. There are several kinds of licence, and they differ in what you may do with the model commercially; for a serious company deployment it's worth reading the one that applies.

A closed model exists only at the provider. You talk to it through an API or through the provider's app. You can't download it, you can't run it offline and you can't stop the provider replacing it or switching it off tomorrow.

The difference isn't in quality. The difference is in who has their finger on the trigger.

Three questions that decide for you

Before you set about comparing benchmarks, run the task through three questions. Usually the very first one gives you the answer.

1. How sensitive is the data?

If you'd mind a stranger reading that text, the task belongs at home. Internal meeting notes, client materials, health matters, salaries, contracts in progress. Not because providers are suspect — but because with sensitive data the strongest protection is when it simply doesn't go anywhere.

2. What does a mistake cost?

When the model gets something wrong in a draft text, you rewrite a sentence. When it gets something wrong in a calculation that goes into a quote, it costs money. The more expensive the mistake, the more you want the strongest model — and above all a human check.

3. How long and complex is the task?

One question and one paragraph of answer is within reach of almost any model. A fifty-page document, a multi-step brief or a change across a large project is a discipline in which the top closed models still have a noticeable lead.

What goes where in practice

Notice that the local model wins in two of the four quadrants — and they are the two in which most daily work lies. That is the whole turnaround that brought local AI back into the game.

Where open models have moved on in 2026

Three things have changed compared with two years ago:

• Small models have got stronger. A model with eight billion parameters now does things that forty used not to be enough for. For summaries, classification and extraction that is fully sufficient.
• There are more families. In Ollama's library you'll find Llama, Qwen, Gemma, Mistral, DeepSeek and open-weight models from OpenAI — plus specialised variants for code. You have enough to choose from by task, not just by brand.
• They can call tools. Newer open models handle structured output and function calling, which is a precondition for anything agentic. Without it a local model would remain just a clever chat.

Where closed models still lead

Let's be frank, because enthusiasm does harm here:

• The hardest reasoning. Multi-step tasks where the model has to go back and correct itself still belong to the top tier.
• Long context. Holding the thread across hundreds of pages is a discipline where the difference is large.
• Being up to date. Closed models have built-in search and more recent knowledge. A local model doesn't know what happened last week — and if you don't tell it, it will happily make it up.
• Images, audio, video. Multimodal abilities are weaker in open models.
• Support and accountability. When something goes wrong, with a provider there is a contract and someone to call.

The best answer is “both”

Almost nobody has just one. A real setup that works looks like this:

1. The local model as the default. Everything that can be done at home is done at home. It's free, without limits and without sending data anywhere.
2. The cloud as escalation. When the local model can't manage the task, or the task is plainly beyond it, it gets sent up.
3. A gateway that decides. So that nobody has to switch between apps by hand. That is exactly what the whole next track of the series is about.

This model is also easy to defend when it comes to regulation: in the EU, transparency rules under the AI Act apply from August 2026, and in general the pressure to know where data flows is growing. When a company has it written down that “sensitive tasks run locally, the rest at provider X”, that is an answer. “We use AI” is not an answer.

Pro tip

Run your own mini-test. Take five tasks you really do often and put them through both a local and a cloud model. Not a benchmark, but your own work. Almost everyone finds after this exercise that for three of the five they can't tell the difference — and those three are exactly the ones that belong at home for good.

That ends the track on local AI. The next one starts with the question of how to get all those models — home and cloud — behind a single switch.