Future TechnologyNPU vs GPU vs CPU: What Actually Runs Your AI
Key takeaways
- A CPU handles logic, a GPU handles flexible parallel maths, an NPU handles one narrow shape of maths very efficiently
- A TOPS figure is a theoretical peak at low precision and cannot be compared across vendors
- Memory bandwidth, not TOPS, sets how many tokens per second you actually get
Every laptop spec sheet sold in 2026 carries a TOPS figure, and almost nobody selling you the laptop can explain what it measures. The NPU vs GPU vs CPU question sits underneath every one of those purchases, and the answer is less about which is fastest than about which is doing the work.
What each unit is actually good at
A CPU has a handful of powerful, flexible cores. It is brilliant at branching logic, sequential work and anything unpredictable. It will run a model, slowly, and it runs everything else on the machine while it does.
A GPU has thousands of simple cores doing the same operation on different data. That is exactly the shape of a matrix multiplication, which is exactly what a neural network is made of. Flexible, fast, and thirsty.
An NPU is fixed-function silicon built for one narrow job: low-precision matrix maths, at very low power. It is faster per watt than a GPU on the work it was designed for and awkward at anything outside that. That power figure is why NPUs appear in laptops and phones rather than servers.
What a TOPS number does not tell you
TOPS means trillions of operations per second. It is almost always quoted at INT8 or lower precision, and it is almost always a theoretical peak rather than anything you will sustain. It does not tell you the precision, whether the figure counts the NPU alone or the whole chip, or whether your software can reach the hardware at all.
Comparing TOPS across vendors is close to meaningless. Two chips with the same number can differ by a wide margin on the same workload, which is worth remembering when reading the Snapdragon C machines or Intel Panther Lake parts against each other.
Why memory bandwidth matters more
Generating a single token requires streaming the model weights through the compute unit. An 8GB model means 8GB moved, per token, every token. That makes memory bandwidth the real ceiling on local inference, not raw compute, which is why our memory types explainer is more useful for predicting speed than any TOPS figure.
Why the M6 split makes the point
Apple gave the entry M6 a dual 16-core Neural Engine and the more expensive M5 Pro a single one. That reads backwards until you notice who buys each. Pro buyers grade video and compile code, which leans on CPU and GPU cores. The entry Mac mini is being sold as a local AI box, so it got the NPU.
The takeaway for a spec sheet: work out what you are asking the machine to do, then look at the unit that does it. The headline number is rarely the one that matters.