Windows with Nvidia: the second route to your own AI

If you have a gaming PC at home, you may have a better machine for local AI than someone who is just buying a Mac mini. An Nvidia graphics card computes inference very fast. There is one condition, and it is uncompromising.

The only number that matters: VRAM

Unlike Macs, where memory is shared, a graphics card has its own memory — VRAM. And there is a hard boundary:

• The model fits into VRAM entirely → only the graphics card computes → very fast.
• The model doesn't fit → part of it is pushed into system memory and the processor computes → an order of magnitude slower.

It is not a smooth transition. It is a cliff. A model that is half a gigabyte short of fitting into VRAM can be five times slower than one that fits with room to spare.

The quickest way to find out how much VRAM you have in Windows is this: Ctrl + Shift + Esc → the Performance tab → GPU → the “Dedicated GPU memory” row.

The numbers are rough — it depends on the quantisation and on how long a context you give the model. Always leave yourself at least a gigabyte of headroom; the context is stored in VRAM too, and it grows with the length of the conversation.

Installation: nothing special

The good news is that on Windows today there is nothing to tune.

1. Update the Nvidia driver. Through GeForce Experience or from the manufacturer's website. This is the one step people skip and then wonder why the model runs on the processor.
2. Install Ollama (the Windows installer) or LM Studio. Both find the graphics card by themselves.
3. Run a model with the same command as anywhere else:

ollama run qwen3

4. Check that it runs on the graphics card. While an answer is being generated, Task Manager must show load on the GPU, not the processor. If only the processor moves, the model didn't fit into VRAM — take a smaller variant.

Ollama can also tell you directly how the model is placed:

ollama ps

In the placement column you'll see whether it runs entirely on the GPU or is split between GPU and CPU. The latter is a signal to reach for a smaller model or stronger quantisation.

Settings for round-the-clock operation

If the PC is to act as a home server, it needs the same adjustments as the Mac in the previous part.

Disable sleep. Settings → System → Power & battery → Screen and sleep. Set sleep to “Never” when the computer is plugged in.

Turn on automatic start after a power cut. This is set in the BIOS/UEFI, usually an item called “Restore on AC Power Loss” or similar. Without it the machine stays off after an outage.

Let the service listen on the network. On Windows too, Ollama starts for localhost only. Add a system environment variable:

OLLAMA_HOST = 0.0.0.0:11434

You set it in Settings → System → About → Advanced system settings → Environment Variables. Then restart Ollama (the icon in the notification area → Quit, and start it again).

Allow the port in the firewall — for the private network only, never for the public one.

And once more the warning from the previous part, because it matters more than everything else: do not forward the port on your router. Ollama has no login. Handle remote access with a private network; see the part on the safe way home.

When Windows is a better choice than a Mac

• You already have a powerful graphics card at home. Zero entry cost compared with buying a new machine.
• You also want to train or fine-tune something. The ecosystem around Nvidia is considerably richer here.
• You want to generate images. Image generation tools have the best support on Nvidia.
• You need raw speed on a model that fits into VRAM.

When a Mac is the better choice instead

• You want a big model, not a fast one. 64 GB of unified memory can handle what an ordinary gaming card wouldn't manage in a month of Sundays.
• It will sit in the living room. A gaming PC is noisy and guzzles more; the difference in both noise and consumption is dramatic.
• You don't want to look after it. A Mac mini runs for months without intervention.

An honest comparison of power consumption

A gaming PC draws tens of watts when idle and easily two to three hundred while generating an answer. If you leave it running round the clock, it shows on the bill — unlike a Mac mini, which is practically free in this respect. When you work out whether local pays off (the part on costs in the third track), this number belongs in the calculation.

A practical compromise that proves itself: let the PC sleep and wake it over the network (Wake on LAN) when it has something to compute. It saves a lot of electricity at the price of a few extra seconds of waiting.

What to watch out for

• Don't look at the core count or the card's clock speed. VRAM decides what you can run; everything else only decides how fast.
• Shared memory is not VRAM. Windows reports “Shared GPU memory”, which is borrowed system memory. A model that ends up there will run slowly.
• Close the game. A game and a model fight over the same memory; when the card has no room, the model is shunted into RAM and slows down.
• Watch the temperatures. Round-the-clock inference is a different load from gaming — more sustained and less bursty. Check that the rig keeps reasonable temperatures even after hours.

The next part explains the word that kept coming up in both previous parts: quantisation. It is the main lever for getting a model into the memory you have.