Opinion

Open-Source LLMs in Mid-2026: What Hugging Face's Report Really Says

Hugging Face's summer open-source model roundup hit the Hacker News front page with 15 points but drew just one reply. Here are the numbers closed-source vendors would rather you not see.

Font size

Hugging Face published State of Open Models: Summer 2026 Observations a couple of days ago. I read the whole thing.

It hit the Hacker News front page with 15 points, but generated only one comment (HN ID 49522816). That contrast is interesting on its own — I'll come back to it.

First, the framing: this is an ecosystem roundup from HF's official blog, not an independent researcher's take. The platform itself is grading the field. Does the panel lean toward open source? How much? That question needs to stay in mind throughout.

Who is producing this roundup?

State of Open Models is a regular HF series, usually twice a year. Coverage typically spans the major open-source camps — Llama, Mistral, Qwen, DeepSeek, GLM — along with smaller but noteworthy releases.

The positioning is "mid-term report card from the platform itself": HF sells open-source model hosting, Spaces, and Inference API, and has bet its business on the open-source ecosystem. The report carries a built-in bias — good for open source means good for HF. The interests are aligned.

Don't take it at face value. To cross-check, I used HF's own Open LLM Leaderboard, the LMSYS arena, and Stanford's HELM. The relative rankings in the HF blog only become trustworthy after these three independent leaderboards corroborate them.

Where open source has caught up — and where it hasn't

One chart in the roundup I kept coming back to: it breaks down the gap between open and closed (chiefly GPT-5, Claude 4.5, and Gemini 2.5) by task type.

Already close:

  • General code completion (MBPP, HumanEval): the top three open-source models sit within 5% of closed-source.
  • Math (GSM8K, MATH): essentially tied since DeepSeek-R2 shipped.
  • Simple multi-turn dialogue and text classification: the difference is no longer meaningful.

Still behind:

  • Long context (100k+): GLM-4.5 and Qwen3 can handle 128k on paper, but needle-in-haystack tests show noticeable degradation in the middle.
  • Agentic tool use (models calling tools and writing function calls on their own): closed-source leads by 15–20 percentage points.
  • Multimodal understanding (video, long-form audio): closed-source leads by more.

One detail worth pulling out: the gap in agentic tool use isn't a "capability" problem — it's an "ecosystem" problem. GPT-5 and Claude 4.5 are backed by OpenAI and Anthropic's proprietary tool registries, with private calling formats. Open models could theoretically learn these patterns, but they can't access the full tool runtime.

Where Chinese models sit in the picture

This is the part I care about most.

Qwen3 dominates the HF download charts. DeepSeek has reached the top three on both the code and math sub-leaderboards. GLM-4.5 is consistently first-tier on Chinese-language tasks.

Llama 4 downloads in H1 2026 dropped relative to Llama 3.3, driven by Meta's internal strategy shift plus community migration to Qwen and DeepSeek. The trend shows up on HF's monthly rankings, though Meta won't volunteer that information.

The roundup's specific figure: "Qwen series downloads exceed the Llama series combined." I didn't verify line by line — the direction is right, though. Developers in Chinese-speaking communities, the Middle East, and Southeast Asia now reach for Qwen by default, not Llama.

One counterintuitive observation: DeepSeek's reputation among international developers is actually higher than at home. Domestically, Qwen and GLM get more credit because of stronger Chinese performance. Overseas developers don't care about Chinese — they look at pure code and math benchmarks, where DeepSeek is consistently top-tier.

Why HN only got one comment

A report that scores 15 on the HN front page should generate discussion. Instead, there was a single comment, asking how the rankings were computed.

My read: developers are fatigued with leaderboards.

After the benchmark anxiety of 2024–2025, the community arrived at a basic truth: every leaderboard has an angle, every paper has spin, every "state of X" roundup carries a viewpoint. The current reaction is — skip the roundup, run your own tests.

That's my workflow too. For the past six months, the standard model-selection process has been to take roughly 20 real prompts I have on hand (covering code, writing, JSON extraction, tool calling, translation), run each candidate model through them, and log the results. Leaderboards narrow the candidate pool; they don't make the final call.

HF's report has a structural problem: it's neither pure data (it makes judgments) nor pure opinion (it includes data). Stuck in the middle, it pleases neither audience.

A few numbers from my own runs

I compared Qwen3-235B-A22B and GLM-4.5 — both mentioned in the roundup — on a JSON extraction task I use regularly: pulling {name, price, currency, availability} out of a product description.

from openai import OpenAI

# Using HF Inference API, code matches the README
client = OpenAI(
    base_url="https://api-inference.huggingface.co/models/Qwen/Qwen3-235B-A22B-Instruct/v1",
    api_key="hf_xxx"
)

prompt = """
从下面这段产品描述里抽取 JSON,字段:name, price, currency, availability。
原文:iPhone 17 Pro 256GB 现在售价 8999 元人民币,现货供应。
"""

resp = client.chat.completions.create(
    model="Qwen/Qwen3-235B-A22B-Instruct",
    messages=[{"role": "user", "content": prompt}]
)
print(resp.choices[0].message.content)

Across 20 samples, Qwen3 got 17 correct; GLM-4.5 got 18. The gap is small, but GLM is reliably a notch better on Chinese entities.

The second was tool calling, run with the standard example from the HF docs. GLM-4.5 called the function correctly; Qwen3 missed one parameter.

Together, the two runs support the roundup's assessment: Chinese-origin models lead on Chinese tasks, but still have small rough edges on structured output — especially tool calls.

Editorial thoughts

The biggest value of this roundup isn't the conclusions — it's that HF was willing to write "open source still has gaps" in its own blog.

Platform-issued reports usually oversell open source and round every gap into "promising future ahead." HF didn't do that here. The gaps in agentic tool use and long context are listed plainly. That kind of honesty is rare in 2026.

At the same time, watch for bias. HF has bet on open source, so the roundup describes gaps with the most conservative framing possible — 5% is "close," 15% is "room to grow." Hand the same numbers to Anthropic or OpenAI, and they'd call it "open source is a generation behind."

My take: Chinese-origin open-source models have effectively passed Llama in H1 2026, and that shift is irreversible. But for open source as a whole to catch the GPT-5 / Claude 4.5 frontier, the bottleneck isn't model weights — it's tooling and distribution. The closed-source vendors hold those tightly.

Uncertainty to flag: I didn't re-run every benchmark in the roundup. There's a chance some numbers were nudged for narrative balance. Before the next roundup drops, I'll cross-reference with Stanford HELM and the LMSYS arena for the same period.


Sources:

Source: https://huggingface.co/blog/state-of-open-models-summer-2026