Running AI on Your Own Laptop: What Local Models Are Actually Good For

There's a moment that happens to almost everyone who uses AI regularly. You're about to paste something into a chat window — a draft contract, a chunk of a client's codebase, a page of notes from a doctor's appointment — and your finger hovers over the keys. Should I be sending this somewhere?

Most of the time you send it anyway, because the tool is useful and the alternative is doing the work yourself. But that hesitation is worth paying attention to, because there's now a real answer to it: you can run a capable AI model entirely on your own laptop, with no account, no internet connection, and no data leaving the machine. What used to require a workstation and a lot of patience now takes about twenty minutes and a free download.

This is a plain-English tour of what that actually looks like — what it's good at, where it falls short, and how to try it without breaking anything.

Why local models suddenly became practical

Two things changed at roughly the same time, and together they moved local AI from "fun weekend project" to "genuinely useful."

The first is that models got dramatically smaller without getting proportionally dumber. A few years ago, useful meant enormous. Today a 7- or 8-billion-parameter model — small enough to sit comfortably on a consumer laptop — handles summarizing, rewriting, answering questions about a document, and light coding at a quality that would have seemed impossible from something that size. It isn't equal to the largest cloud models, and anyone telling you otherwise is selling something. But for a large slice of everyday work, the gap has narrowed enough that it stops mattering.

The second change is quantization, which sounds technical but is easy to picture. A model's "weights" are just an enormous pile of numbers. Quantization stores those numbers at lower precision — think of saving a photo as a high-quality JPEG instead of a raw file. You lose a little fidelity, and in exchange the file shrinks by roughly 60–75%. That single trick is what makes an 8 GB laptop a viable place to run an AI model at all.

Local AI didn't win by getting bigger. It won by getting small enough to fit where you already are.

Add to that the shift in laptop hardware — Apple's unified memory architecture in particular lets the graphics side of the chip reach the same memory pool as the processor — and a mid-range machine from the last few years is now a perfectly reasonable place to run a model.

What you actually need

The honest hardware answer is: memory matters more than anything else. Not your processor speed, not your storage, not whether you have a gaming GPU. Memory.

A useful rule of thumb: a quantized model needs roughly one gigabyte of memory per billion parameters, plus a bit of headroom. So an 8B model wants about 8–10 GB free. If you try to run something too large, it won't explode — it will just get painfully slow as your system starts swapping to disk. That slowness is the signal to drop down a size.

You'll also want 10–30 GB of free storage, because models are big files and you will inevitably download three of them to compare.

Two tools, two personalities

Almost everyone ends up with one of two pieces of software, and the choice comes down to how you like to work rather than which is better.

Ollama is the command-line option. You install it, type ollama run llama3.2, and it downloads the model and drops you into a chat prompt in your terminal. Its real strength is that it quietly runs a local API server in the background, so your own scripts can talk to it the same way they'd talk to a cloud provider:

# Pull a model and chat with it
ollama pull llama3.2
ollama run llama3.2

# Or hit it from code / a script
curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "Summarize this in three bullet points: ..."
}'

If you write code, this is the one. Swapping a script from a paid API to a local model becomes a matter of changing a URL.

LM Studio is the graphical option. It looks like a normal desktop app: browse a catalog of models, click download, chat in a window. It shows you how much memory each model will need before you commit, which spares you a lot of trial and error. If the phrase "open a terminal" makes you tired, start here.

Both run the same underlying models. Neither locks you in. Installing one and then switching costs you nothing but a re-download.

Where local models genuinely shine

The most compelling use isn't raw capability — it's the class of work you'd rather not upload.

Anything confidential. Client documents, internal financials, medical paperwork, unreleased code, HR material. If you've ever closed a chat window rather than paste something in, that task belongs on a local model. The privacy isn't a policy promise; it's architecture. There's no network request to make.

Anything repetitive and high-volume. Cleaning up a thousand product descriptions, tagging a backlog of support tickets, converting messy notes into consistent formats. Per-token pricing makes this genuinely expensive in the cloud. Locally, the marginal cost of the ten-thousandth request is the electricity to run your laptop for another few minutes.

Anything offline. Flights, trains, rural cabins, conference venues with hostile Wi-Fi. A local model doesn't care.

The question isn't "is this as good as the best cloud model?" It's "is this good enough for this specific job, and is that job one I'd rather keep on my own machine?"

Where they still fall short — and it's worth being honest

A local 8B model is not a frontier model, and pretending otherwise leads to disappointment.

Complex reasoning is weaker. Multi-step problems, subtle logic, and long chains of inference are exactly where the big cloud models earn their size. If a task requires holding many constraints in mind at once, you'll feel the difference immediately.

Knowledge is frozen and shallower. Local models are smaller, which means less packed in, and they have no live web access unless you wire that up yourself. For anything current, they'll confidently make things up. Verify anything factual — this matters most for health, legal, and financial questions, where a plausible-sounding wrong answer does real damage.

Long documents are harder. Smaller models generally handle less context, and stuffing a huge document in will either fail or quietly degrade the quality of the answer. Chunking works, but it's manual effort the cloud tools handle for you.

Speed depends entirely on your machine. On a recent Apple Silicon laptop, responses feel close to instant. On an older Intel machine without a capable GPU, you'll watch words appear one at a time. Try it before you build a workflow around it.

The practical conclusion most people land on isn't "replace the cloud." It's use both, deliberately: local for the private, repetitive, and offline work; cloud for the hard thinking. That split costs nothing and covers more ground than either alone.

A twenty-minute first try

If you want to test this today, keep the first attempt small so a bad result doesn't sour you on the whole idea.

1. Install one tool. LM Studio if you want buttons, Ollama if you want a terminal. Don't install both.
2. Start with a small model. A 3B or 7B quantized model. Yes, a bigger one exists. Start small anyway — you're testing whether the setup works, not benchmarking.
3. Give it a real task you already know the answer to. Summarize an email you wrote. Rewrite a paragraph. Explain a snippet of your own code. Comparing against something familiar tells you far more than a generic test prompt.
4. Watch your memory and your patience. If it's crawling, drop a size. Speed you can live with beats quality you'll never wait for.
5. Only then size up. Once you know the workflow fits, try a larger model and see whether the extra wait buys enough extra quality to be worth it.

Total time: about twenty minutes, most of it spent downloading.

The quiet shift underneath all this

What makes local AI interesting isn't that it's free, though it is. It's what it does to the default.

For the last few years, using AI has meant sending your work to someone else's computer and trusting the arrangement. That was a reasonable trade when it was the only option. It's no longer the only option — and once a decent model is sitting on your own hard drive, the question flips. Instead of "is it safe to send this?", it becomes "does this particular task actually need the big model?"

Often, the answer is no. And a tool that runs on your laptop, needs no account, works on a plane, and never sends your draft anywhere turns out to be surprisingly hard to give up.

In short: smaller models and quantization have made local AI genuinely practical on ordinary laptops. Memory is the constraint that matters; 16 GB is the comfortable middle. Ollama suits people who write code, LM Studio suits everyone else. Local wins on privacy, repetition, and offline work; the cloud still wins on hard reasoning and current information. Use both on purpose.

It's a good moment to be curious. The download is free, the setup is short, and the worst outcome is that you learn something about your own laptop.