Future TechnologyWhat you can actually run locally in 2026, by how much RAM you have
Key takeaways
- Memory is the binding constraint for local AI, not compute. 64GB is a strong target and 128GB unlocks the largest models.
- 8GB of VRAM handles 3B to 8B models, 12GB covers 13B comfortably, and 16GB reaches 70B parameters.
- Q4_K_M quantization cuts a model to roughly a quarter of its size with minimal quality loss, moving most models down a full hardware tier.
- For code, DeepSeek Coder and CodeLlama beat general purpose models of the same size.
Memory is the binding constraint on local AI, not raw compute. That one fact decides most of what follows, and ignoring it is how people end up with the wrong graphics card. Here is what fits, tier by tier.
Entry level: 8GB VRAM and 32GB system RAM
An RTX 4060 Ti with 8GB alongside 32GB of system memory handles 3B to 8B models comfortably. Qwen 3.5 7B is the current standout at this size, with hybrid thinking modes that switch between quick answers and slower reasoning depending on the question. Llama 3 8B, Mistral 7B and Phi-3 Mini are the other reliable picks.
This tier covers summarising, drafting and light code completion. It does not cover long context work, and that is the limit you will hit first. Laptop buyers should also watch what the newer mobile silicon does here, since Intel's Panther Lake parts shift the entry point for machines without a discrete card.
Mid range: 12GB VRAM and 32GB DDR5
An RTX 4070 Super with 12GB paired with a 32GB DDR5 kit is the sweet spot for most people. It runs Llama 3 8B and 13B smoothly, handles Code Llama 13B, and will manage Mixtral 8x7B with partial offloading to system RAM. Offloading costs you speed, so treat it as a ceiling rather than a configuration you live in.
High end: 16GB and above
An RTX 5070 Ti with 16GB and 48GB of system RAM reaches 70B parameter models. An RTX 5090 with 32GB and 128GB of system memory opens up 405B inference. The 24GB cards, from the 3090 through to the 5090, remain the practical sweet spot for anyone not building a rack.
Apple silicon is the quiet alternative. Unified memory means a Mac mini M4 with 24GB punches above what the number suggests, because there is no split between VRAM and system RAM to manage.
The two rules worth memorising
Memory decides whether a model loads at all. Compute only decides how fast the tokens arrive once it has. So aim for 64GB as a working target and treat 128GB as the threshold for the largest open weights.
Q4_K_M quantization cuts a model to roughly a quarter of its size with minimal quality loss. It is the sensible default for real use, and it moves most models down a full hardware tier. A 70B model that will not fit at full precision becomes workable quantized, which is usually cheaper than buying a bigger card.
Storage and the rest of the build
You will collect more model weights than you expect. A fast NVMe drive matters more than people assume here, because slow load times make experimentation tedious enough that you stop experimenting. Buy capacity before speed bins.
For code specifically, DeepSeek Coder and CodeLlama beat general purpose models of the same size. A 13B coding model will usually outperform a 30B general model on the work you actually run.
What this saves you
The gap between what people think they need and what they need is usually one graphics card, and that card is expensive right now. Memory and storage prices have moved sharply this year, partly because AI infrastructure spending is absorbing supply that used to reach consumers. The cheaper move is to decide the largest model you genuinely want to run, apply the quantization rule, and buy for that number rather than the one on the box. If you are curious how local models compare with the hosted frontier, the published benchmark numbers set a useful floor for expectations.
Some links in this article are affiliate links. We may earn a small commission at no extra cost to you.