What runs where, and who may start it
The question is no longer “ChatGPT, yes or no”. The question is what runs where and who may start it. Local and in-house AI is growing alongside the cloud: a Mac mini handles inference, open weights keep closing the gap, and agents are the product of the year and the risk of the year. This series takes you from your first local model to a control room you run the whole thing from — and to the rules without which it all falls apart.
45 parts · six tracks · more coming
Read it front to back like a course, or jump around by tag. If in doubt, start with the first part of each track — they assume you have installed nothing yet.
Prefer a fixed order to a signpost? Follow the path An agent on your NAS from scratch (13 steps).
01 · 5×
Model, harness, agent — and why the question moved from “AI yes or no” to “what runs where”.
02 · 5×
Two complete setups from hardware to invoice: what runs where, what it costs and why it is put together this way.
03 · 5×
Your first model on your own machine: Mac mini, Windows with an Nvidia card, memory, and when local is enough.
04 · 6×
One gateway in front of every model, API keys in one place, and routing each job to what handles it best.
05 · 8×
A control room in your pocket: Telegram as the panel, a safe route home, and a queue you approve yourself.
06 · 8×
Files from your disk into the chat: how it really works, what RAG is, what MCP does, what stays off the cloud.
07 · 8×
Agents are the product of the year and the risk of the year. One page of rules and the case studies behind them.
Tags
13 parts tagged “Local AI” · Clear filter
Five accounts, three subscriptions, every model in a different tab and the files still sitting on disk. This part is the map of the whole series: what a control room is, which four layers it consists of and in what order to build it.
“Do we use AI?” is a question from 2023. Today everyone gives the same answer and it tells you nothing. The useful question has two halves — where that particular task runs and who is allowed to start it — and the answer fits on one page.
A specific person, specific hardware, specific accounts. What runs where, what can be cancelled, when it pays to move from a subscription to the API — and in what order to build it so that each step makes sense on its own.
No account, no card, no data sent to anyone. Install one app, run one command, download one model — and an hour from now you have AI on your own machine that works even on a plane.
A quiet box next to the router that runs all the time and answers your questions from your phone and your laptop. How to turn a Mac mini into an inference server, how much memory to buy and what to do so it doesn't switch itself off.
A gaming PC is often a better machine for local AI than a Mac — as long as the model fits into the graphics card's memory. How to find out what your rig can handle, what to set up in Windows, and why VRAM is the only number that matters.
Why the same model comes in five different sizes in the library, what Q4 in the file name means and how to work out whether it will fit in your machine. One table, one formula, no maths.
Open models are no longer the poor man's option. For half of everyday work you won't notice the difference — and for the other half you'll notice it a lot. How to decide task by task instead of picking a camp.
A local model isn't free and the cloud isn't expensive — until you start counting volume. How to work out your own break-even point, where the costs nobody tells you about are hiding, and why money isn't the main reason for going local anyway.
You type a message on your phone, your model at home works it out, and the answer comes back into the chat. A complete guide: creating the bot, where the token belongs, how to allow only yourself, and the fifty lines of code that tie it all together.
Forwarding a port on your router is the fastest way to hand your model to the entire internet. A private network makes it available only to your devices and takes ten minutes to set up — no public IP, no certificates, no holes in the firewall.
“AI sees my files” means five different things with very different consequences for privacy. The difference between attaching, an index, a connector and direct access — and why you should know it before you upload a contract anywhere.
Uploading a thousand documents to a model and expecting it to know them doesn't work. How search over your own material really operates, why it most often fails on bad data, and what to do to make it answer correctly.
Get the next part
New parts and what matters in the world of agents go out in the weekly newsletter.
The Top 30 tips e-book, free — it lands in your inbox right away.
Then 1 tip a week · no spam · unsubscribe in one click