Open vs Closed: Reading the Four-Month Gap

In 2023 the gap between the best closed AI models and the best openly available ones was measured in years. GPT-4 had no open rival, and the strongest open models were clearly a generation behind. By late 2026 that gap is measured in months, and on some leaderboards in weeks. This is one of the most consequential shifts in the AI industry, and it is widely misunderstood in both directions. Enthusiasts declare that open models have caught up. Sceptics point to the hardest benchmarks and declare they never will. Both are reading a single number when the picture has several dimensions.

The headline number

The most rigorous estimate comes from Epoch AI, which tracks model capability using a composite index across many benchmarks. In a May 2026 analysis, it found that since the start of the year the most capable open-weight models had lagged the closed frontier by an average of about four months, equivalent to an eight-point gap on its index. For calibration, Epoch noted that is roughly the distance between two consecutive flagship releases from a single leading lab.

Two caveats from the same analysis matter more than the number itself. First, closed labs keep their most advanced models unreleased for a period, so the public frontier understates what closed labs actually have. Second, public benchmarks may flatter open models, since publicly known test sets are easier to optimise against. Both effects push in the same direction: the true gap is probably somewhat larger than four months.

Snapshots from Artificial Analysis, which runs its own independent evaluations, tell a consistent story with more volatility. (Its index was rebuilt during the year, so absolute scores are not comparable across versions; the gaps are.) In late April 2026 the best open models sat within three to six points of the leading proprietary model. By late September, after a burst of closed releases, the gap had widened to more than ten points. Then on 23 October, Moonshot's Kimi K3.1 Pro landed three points behind the leader, the highest score an open-weight model had recorded. Four days later, Artificial Analysis replaced its index with v5, in which 60 percent of the tests are private, and the same comparison became 46 against 58: a 12-point gap. K3.1 Pro's weights were published on 4 November. The gap breathes. It widens when closed labs ship and narrows when open labs answer. It also widens when the tests become harder to optimise for, which is a useful reminder that some of the apparent convergence has been benchmark convergence. The trend over two years has still been toward closing the gap.

The gap is not one gap

The more useful finding is that the lag differs sharply by capability.

Small or vanishing gap: - General conversation and writing quality, where human-preference leaderboards show the top open and closed models separated by margins that most users would not notice. - Standard knowledge and comprehension benchmarks, where open models had effectively matched the frontier by early 2026. - Summarisation, extraction, classification and translation, which constitute most of the AI work actually done inside companies.

Moderate gap: - Coding on well-specified tasks, where open models are strong but trail on the most complex repositories. - Multimodal understanding, which open models added later and are still refining.

Large gap: - Long-horizon agentic work. In Artificial Analysis's April data, the leading open models scored in the low-to-mid forties on a hard agentic coding and terminal benchmark, against around sixty for the best closed model. Tasks that require planning, tool use and recovery from errors over many steps are where frontier models still earn their price. - Hard reasoning. On research-level physics problems, the same review found open models far behind. - Cyber capability. Epoch's October 2026 analysis put the best open model six to seven months behind the closed frontier on its cyber-specific index. Some of this is deliberate: Moonshot, for instance, removed cyber-related training data from its open release of Kimi K3.1 Pro, accepting a measurable drop in that domain to avoid releasing offensive capability. - Spatial and visual reasoning. On an Epoch benchmark based on assembling furniture from photographs, open models trailed by about seven months. - Knowing what they don't know. Artificial Analysis's knowledge-and-hallucination measure showed the clearest gap of all in April 2026: leading closed models scored well into positive territory, while the best open models clustered near zero and one prominent model scored negative, meaning it hallucinated more than it answered correctly on hard factual questions.

This last point deserves emphasis for anyone deploying AI in professional settings. Calibration, the ability to say "I don't know" rather than inventing a plausible answer, is less visible than raw intelligence and more important for trust. It is also an area where retrieval-based designs, which ground answers in supplied documents, compensate substantially.

What closed models still do better

Setting benchmarks aside, closed frontier systems hold several practical advantages:

• The hardest problems. If your task sits at the edge of what AI can do, the frontier is where it gets done, and four months of progress can be the difference between failing and succeeding.
• Factual reliability. Better calibration and broader knowledge reduce the burden on human review.
• Integrated products. Closed providers ship models together with search, code execution, file handling, agents, enterprise administration and support. Assembling the equivalent around an open model is real engineering work.
• Safety infrastructure. Abuse monitoring, red-teaming and policy enforcement are run by the provider. With open weights, that responsibility is yours.
• Zero operations. No hardware, no capacity planning, no inference optimisation. You send a request and pay for what you use.

What closed models miss

The weaknesses of closed models are mostly structural rather than technical, which is why they do not close as capability improves:

• No custody. Your data goes to the provider. Contracts can limit what they do with it, but cannot change which governments can compel them.
• No version control. Models change and are retired on the provider's timeline. Validated behaviour can drift without notice.
• No inspection. You cannot examine the weights, and in most cases you know little about the training data.
• Pricing power. The provider sets the price and can change it. There is no competing host for the same model.
• Limited adaptation. Fine-tuning, where offered, happens on the provider's platform with your data on their infrastructure.
• Single points of failure. An outage, a policy change or a geopolitical restriction affects every customer at once.

What open models do better

• Custody and sovereignty. Run anywhere, under any jurisdiction you choose, with nothing leaving your perimeter.
• Cost at volume. In April 2026, nine of the thirteen models on Artificial Analysis's intelligence-versus-price frontier were open-weight. For high-volume work the savings are large.
• Stability. Pin a version and it never changes until you decide.
• Customisation. Fine-tune on proprietary data without sharing it, and distil large models into small specialised ones.
• Latency and offline use. On-premises and on-device inference avoid network round-trips and work without connectivity.
• Ecosystem. Many providers compete to host the same weights, and a large research community improves tooling around them.

What open models miss

• The last few months of progress, which matter most on the hardest tasks.
• Calibration and breadth of knowledge, as noted above.
• Turnkey tooling and support. You assemble and maintain the system yourself or through a partner.
• Practicality at the very top end. The strongest open models are enormous. Kimi K3.1 Pro has 2.8 trillion parameters in total. "Open" does not mean "runs on your laptop".
• Licensing uniformity. Terms range from fully permissive to restrictive custom licences. Kimi K3.1 Pro's licence, for example, carries clauses governing use as a hosted service.
• Provenance comfort. For much of 2026 the top open-weight models came from Chinese labs, which some regulated buyers cannot use regardless of deployment. Strong Western open releases in October 2026 have widened the choice.

Reading the timeline

Three observations about where this goes.

First, the gap has been remarkably stable. Several analyses over the past two years have found a lag in the range of three to six months, holding through multiple supposed step-changes. Betting on a sudden widening has repeatedly been a losing position.

Second, a fixed lag in time is a shrinking lag in practical value. Four months behind in 2023 meant missing core capabilities. Four months behind in late 2026 means having roughly what the frontier offered in mid-2026, which is more than enough for the large majority of business tasks.

Third, the gap that remains is concentrated where it matters least for most organisations and most for a few: autonomous agents, frontier research, offensive and defensive cyber work. If those are your core use cases, the frontier is worth paying for. If your work is reading, writing, searching and summarising, the four months are mostly an abstraction.

A decision rule

The most practical way to choose is to ask, for each workload: is the frontier's edge worth what it costs here? The cost includes not only the price per token but the loss of custody, the exposure to version changes and the dependency on a single provider.

For a complex research agent working on public information, the frontier's edge is worth it. For classifying ten million internal documents containing client data, it almost certainly is not. Most organisations will find their portfolio splits clearly once they ask the question honestly, and that the open side of the split is larger than they expected.