Future TechnologyFuture Technology
AI

OpenAI's GPT-5.6 Sol Just Ran at 750 Tokens a Second, and It Wasn't on Nvidia

· 4 min read · By Nath Connell

Key takeaways

  • OpenAI previewed GPT-5.6 Sol running on Cerebras wafer-scale chips at up to 750 tokens per second
  • Access is limited to select customers first while Cerebras scales capacity
  • It's a serving decision, not a training one, and it points at Nvidia's grip loosening on inference specifically
  • Faster output changes what a chat interface can be used for in real time, not just its benchmark score

OpenAI gave a small group of customers an early look at GPT-5.6 Sol this week, and the number that got people talking wasn't a benchmark score. It was speed. Running on Cerebras wafer-scale hardware, the model is hitting up to 750 tokens a second. For context, a fast Nvidia-served frontier model today typically tops out well under half that.

Here's the bit that matters more than the headline figure: this is about serving the model, not training it. OpenAI still trains on whatever mix of Nvidia and custom silicon it always has. What changed is who answers your question once the model exists. And for the first time, the answer to that isn't automatically "an Nvidia GPU."

Why raw speed changes the experience

Benchmark scores measure how smart a model is. Speed measures how it feels to use one. At 750 tokens a second, a long answer that used to stream in over several seconds now appears close to instantly. That is the difference between a chatbot that feels like typing to a person and one that feels like a terminal spitting out a completed thought. Most people never read a benchmark chart, but everyone notices lag.

Cerebras builds its chips differently to Nvidia. Instead of cutting a silicon wafer into hundreds of separate dies, it keeps the whole wafer as one giant processor. Data doesn't have to hop between chips to get an answer out, which is a large part of where that speed comes from.

Access is gated, and that's the catch

For now, GPT-5.6 Sol on Cerebras is only available to a select set of customers while capacity scales up. That's the normal pattern for a new hardware partnership: prove it works for a few big accounts, then widen the pipe. Don't expect it in the free tier chat window next week.

Still, the signal is bigger than the rollout size. If a frontier lab is willing to run its flagship model on non-Nvidia silicon in production, at this scale, that is Nvidia's clearest competitive threat yet in the one part of the AI stack it has owned outright: inference.

Why it matters

Training was always going to attract alternative chipmakers eventually, since it is a small number of enormous one-off jobs. Inference is different. It runs constantly, for every user, forever, and it is where most of the real spend actually lives over a model's lifetime. If frontier models can get this fast on someone else's silicon, the inference market stops being a one-horse race, and every other AI lab now has a decision to make about where it serves from too.

Sources