For three years, “which model?” was a defensible thing for a product team to spend a quarter arguing about. As of March 2026, it mostly isn’t.
Stanford’s AI Index 2026 puts the top of the Arena leaderboard like this: Anthropic 1,503, xAI 1,495, Google 1,494, OpenAI 1,481, Alibaba 1,449, DeepSeek 1,424, with four companies clustered inside 25 Elo points (Ch2). The US–China gap, which was a strategic talking point eighteen months ago, has effectively closed; the top US model leads by 2.7%, and the lead has changed hands repeatedly since early 2025.
The report draws the conclusion itself, and it is worth quoting because it is a product strategy in one line: model convergence shifts competitive pressure “toward cost, reliability, and domain-specific performance.”
That sentence should reorganise your roadmap.
Does the model still matter?
For capability, decreasingly. For everything else, yes, though not in the way procurement decks assume.
The convergence is not only at the top of the leaderboard. On professional-domain benchmarks covering tax, mortgage processing, corporate finance, and legal reasoning, models score between 60% and 90%, and the top fifteen models are separated by as little as three percentage points (Ch2). Fifteen models, three points. Any architecture whose value proposition is “we picked the right one” is built on a rounding error.
Two caveats keep this honest. First, the open-weight gap reopened in 2025 after nearly closing: the top closed model now leads the top open model by 3.3%, up from 0.5% in August 2024, and six of the top ten Arena models are closed (Ch2). The “open models will catch up imminently” thesis had a bad year. Second, the benchmarks themselves are shakier than their use implies: one review found invalid-question rates ranging from 2% on MMLU Math to 42% on GSM8K, and separate research suggests Arena standing may partly reflect adaptation to the platform rather than general capability (Ch2).
So: the models are close, and the instruments measuring them are imprecise. Both facts point the same direction. Stop optimising the model choice and start optimising the system.
What “reliability” means when capability is a commodity
The AI Index’s phrase “jagged intelligence” is the most useful mental model in the report. Gemini Deep Think earned a gold medal at the 2025 International Mathematical Olympiad, working end to end in natural language inside the 4.5-hour limit. The top model reads an analog clock correctly 50.6% of the time, against 90.1% for humans (Ch2).
That is not a contradiction to be resolved. It is the shape of the technology: extraordinary on some tasks, arbitrarily poor on adjacent ones, with no reliable intuition about which is which. Which means the engineering question is never “is the model good?” but “is it good at this, measured on our data, and what happens on the fraction where it isn’t?”
Three numbers set the design constraints:
- Agents fail roughly one attempt in three on structured benchmarks, even after improving from 12% to 66.3% on OSWorld in a single year (Ch2).
- Hallucination rates span 22% to 94% across 26 top models on a knowledge-versus-belief benchmark, and models degrade sharply when a false premise is framed as the user’s own belief (Ch3).
- Safety degrades across every model tested under adversarial prompting, even where standard-use ratings are “Very Good” (Ch3).
None of those are solved by upgrading the model. They are absorbed by the system around it.
The system around the model
If capability is converging and reliability is the differentiator, then the differentiating engineering is unglamorous and specific.
Evals on your data, in CI. Public benchmarks tell you almost nothing about your domain; that is the direct implication of fifteen models within three points on professional tasks. A few hundred labelled examples from your actual workload, run as a gate on every prompt change and every model upgrade, is worth more than every leaderboard combined. It is also the only mechanism that makes provider substitution a decision rather than a gamble.
Grounding you control. Retrieval against your corpus, with citations surfaced to the user, so the answer’s provenance is inspectable. The knowledge-versus-belief data makes this a correctness requirement, not a UX preference.
A designed failure path. At a one-in-three failure rate, “what happens when it’s wrong” is a normal operating condition. Confidence thresholds, human handoff, graceful degradation to the deterministic path.
Router, not monolith. Once models are near-equivalent at the top, per-task routing to the cheapest model that passes your evals is straightforward margin. Cost is now an explicit axis of competition (the report says so), and inference cost is the line item that decides whether a feature survives 10x volume. Our run cost governance post covers the operational side.
Domain specificity as the actual moat. The third axis the report names is the one nobody can copy: your data, your workflows, your evaluation criteria, your users’ definition of a good answer. That is what a competitor cannot buy from a model provider, and it is where the engineering effort should concentrate once the model is a swappable dependency.
An abstraction layer that is thin but real. Not a framework that reimplements every provider feature; those age badly. A boundary that isolates provider-specific calls, so switching is a week rather than a rewrite. We wrote the detailed version in AI portability in enterprise applications.
The vendor risk nobody prices
Here is the argument for portability that has nothing to do with pricing negotiations. Vendor transparency is going backwards: the Foundation Model Transparency Index rose from 37 to 58 between 2023 and 2024, then fell to 40 in 2025, with the widest gaps in training data, compute, and post-deployment impact (Ch3). Documented AI incidents rose to 362, up from 233 (Ch3). Industry produced over 90% of notable frontier models in 2025 (Ch1).
You are building on infrastructure controlled by a handful of companies that are disclosing less each year, deprecating models on their own schedule, and repricing as their own compute costs move. That is a perfectly reasonable thing to build on; it is also a dependency that deserves the same treatment you would give any other single point of failure.
The practical test is simple, and most teams fail it: if your provider deprecated your model next quarter, how long until you are running on another one, and how would you know quality hadn’t dropped? If the answer to the second half is “we’d find out from users,” the eval suite is the missing piece, not the abstraction layer.
What this changes on Monday
Stop treating model selection as a strategic decision and start treating it as a procurement decision with a quarterly review. Move the effort you were spending on it to four things: an eval suite on your own data, grounded retrieval, a designed failure path, and a cost model that survives your growth case.
Convergence at the frontier is good news for anyone building products. It means the hard part is back to being software engineering, and software engineering is a thing you can actually get better at. It also means the gap between organizations with AI in production and the 88% who have merely adopted it is now almost entirely about execution, which we cover in the AI execution gap.
If you are choosing a delivery partner on this basis rather than a model, our vendor evaluation scorecard is the framework we would want to be judged against, and vendor-neutral software development is why we do not have a preferred model to sell you.
All figures cited from the Stanford HAI Artificial Intelligence Index Report 2026 (9th edition). Model rankings and benchmark scores are a March 2026 snapshot and will move.
