Opinion

China's LLM Race Just Shifted Gears: Z.ai Bets Big on Post-Training

Z.ai CEO Jie Tang declared 'Death of Params' on the Latent Space podcast, with GLM 5.3 redirecting R&D spend from pre-training scale to post-training infrastructure, marking the first direct challenge to the 'bigger is better' orthodoxy from a major Chinese lab.

Font size

Jie Tang dropped a line on the Latent Space podcast this week: Death of Params.

This was not small talk. Z.ai (formerly Zhipu AI / 智谱) is committing to a track: post-training scaling. GLM 5.3 is the first production-scale validation of that bet. The significance lies not in any single benchmark score, but in the fact that Z.ai is migrating its R&D budget and engineering focus from pre-training compute to post-training infrastructure. This is a roadmap decision, not a version bump.


What Tang Said on Latent Space

Latent Space ranks among the most-watched AI industry podcasts overseas. Inviting a Chinese frontier-lab CEO onto the show already carries signaling weight.

Tang's core argument can be compressed into three points:

  1. Returns on parameter scale at the pre-training stage have hit a ceiling. Scaling to 1T or 10T parameters yields sharply diminishing marginal gains over the prior generation.
  2. The arena that remains underexplored is post-training: RLHF, SFT, inference optimization, data synthesis. The scaling laws in these stages have not yet plateaued.
  3. The competitive divide over the next 12–24 months will be defined by depth of post-training, not by how many parameters a base model can swallow.

The first point is a diagnosis, the second a prescription, the third a prediction. Read together, they amount to a direct repudiation of the dominant playbook Chinese AI has followed for the past three years: copy whatever OpenAI does.

GLM 5.3 Is a Bet, Not an Upgrade

Casual users will scan GLM 5.3 for benchmark deltas. Scores on MMLU, GSM8K, and HumanEval did improve, though not by screenshot-worthy margins.

What actually matters is how Z.ai built 5.3:

  • Synthetic training data: GLM 5.0 / 5.1 generated high-quality SFT data internally, followed by cleaning, scoring, and filtering. The pipeline is publicly hiring, with more engineering roles than research positions.
  • Multi-stage RLHF: Helpfulness and harmlessness are trained separately, then merged via preference optimization. The reward model uses an ensemble rather than a single scorer.
  • Training-time reasoning: CoT, self-consistency, and process reward are baked into the training loop rather than bolted on at inference.
  • Compute allocation: Post-training consumes over 60% of total training compute, an unusual figure among Chinese frontier labs in 2024.

Now the framing as a roadmap decision makes sense. In GLM 5.3's GPU-hour breakdown, the pre-training share is smaller than in 5.0. While competitors keep shouting about training 1T-parameter models, Z.ai has quietly shifted compute to post-training.

Why This Signal Comes From China

The timing is not coincidental. Viewed against the 2024–2025 compute landscape, the picture sharpens.

Two paths were on the table:

  • Path A: Follow OpenAI, Anthropic, and xAI into 1T+ cluster territory, grinding harder on pre-training.
  • Path B: Accept that compute is an order of magnitude behind and spend on higher-ROI stages.

Top Chinese labs (DeepSeek, Moonshot, Qwen, Zhipu, and others) hold roughly one-fifth to one-third of the H100 / H200 inventory that the top three US companies command. Catching up is not impossible, but the route is expensive, slow, and the leaders keep accelerating.

Tang never spelled this out on the podcast, but the logic is evident: if the frontier players have already pushed the pre-training parameter curve into its S-curve bend, why fight for position on the flat right tail? Switching to a curve that has not yet been flattened, post-training scaling, might actually buy room to lead.

What Post-Training Scaling Actually Means

In engineering terms, post-training scaling is not a slogan. It is a set of concrete moves, each visible in GLM 5.3's training pipeline.

1. Data Synthesis + Quality Filtering

# Simplified GLM 5.3 data synthesis pipeline (pseudocode)
from glm import Generator, RewardModel

base = Generator("glm-5.1")
reward = RewardModel.load_ensemble(["rm-5.1-a", "rm-5.1-b", "rm-5.1-c"])

raw = base.sample(prompts, n=16, temperature=0.9)
scored = reward.batch_score(raw)
filtered = [r for r, s in zip(raw, scored) if s > 0.85]
diverse = deduplicate_by_embedding(filtered, threshold=0.92)
final = quality_filter(diverse, min_tokens=200, max_tokens=4000)

An ensemble of three reward models prevents single-model reward hacking. Deduplication guards against synthetic data collapsing into near-duplicates and distorting the training distribution.

2. Multi-Stage RLHF

Rather than a single reward model trained end-to-end, capabilities are decomposed:

  • Stage 1: helpfulness only (learn to be useful)
  • Stage 2: safety / harmlessness (learn to stay within bounds)
  • Stage 3: reasoning / CoT (learn to externalize the reasoning process)
  • Stage 4: merge + DPO (consolidate preferences)

Each stage is evaluated independently with its own checkpoint. Failure triggers rollback to the previous stage.

3. Training-Time Reasoning (Process Reward)

This is where GLM 5.3 pulls ahead of 5.0. Instead of scoring only the final answer, the model is scored at each step of its reasoning chain. During training, it is forced to learn which step went wrong, rather than getting credit for a correct outcome reached by lucky guessing.

What This Means for Other Chinese Players

If post-training scaling holds up (and the qualifier "if" matters, since this path is still being validated), compute-allocation models across the Chinese AI sector will need rewriting.

A few projections:

  • DeepSeek: Already winning on MoE architecture and training efficiency, but its post-training investment ratio trails Z.ai's. If Z.ai's bet pays off, DeepSeek's narrative needs a new chapter.
  • Moonshot (Kimi): Long-context is a pre-training problem, so post-training offers less leverage there. But agent capabilities and tool-use on the product side must come from post-training. Expect team expansion in that area.
  • Qwen / Tongyi: The open-source playbook leans on community ecosystem, but if the flagship proprietary model's post-training share doesn't keep pace, the risk of falling behind is real.
  • Startups: Strong models are now achievable without a 10K-H100 cluster. Post-training economics are friendly: token-level optimization can suffice without a massive GPU wall.

The bottom line: throwing more GPUs at the problem is no longer the only answer. Engineering depth is becoming more valuable than cluster size. Saying this in 2024 would have drawn laughs. Saying it in 2025 might just trigger the next round of reorgs.

What Hasn't Been Validated Yet

In fairness, Z.ai's path carries real risks:

  • No rigorous data exists on where post-training scaling's ceiling actually sits.
  • Process-reward-trained models might still collapse on long-tail tasks; current evidence is thin.
  • Synthetic data, beyond a certain share of the training mix, may contaminate itself through feedback loops.
  • Buyer behavior is a wild card: many procurement teams still evaluate by parameter count, and GLM 5.3 has to talk them off that metric.

But regardless of how it plays out, the Death of Params slogan has been fired. Z.ai is the first major Chinese LLM company to publicly challenge the "bigger parameters, stronger model" consensus. That signal deserves a serious hearing from every team still grinding toward the next trillion-parameter checkpoint.


Source: Latent Space podcast, Jie Tang interview, https://www.latent.space/p/ainews-death-of-params-zai-ceo-jie

Cover image keywords: cover "death of parameters GPU"

Source: https://www.latent.space/p/ainews-death-of-params-zai-ceo-jie