LLMs

Can Tencent’s Rebuilt Hy4 Architecture Be Reproduced?

Hy4 reached the Hacker News front page, but its official summary omits model parameters, training details, code, weights, and benchmark data. This analysis outlines the reproducibility checklist a model release should meet and what Tencent still needs to disclose.

Font size

Hy4-related content reached the Hacker News front page with 99 points and 29 comments. Popularity is not a technical conclusion. To assess the value of Tencent’s architectural overhaul, researchers need to know whether its training, inference, code, and weights can be reproduced.

Tencent released a research page for the Hy4 model at hy.tencent.ai/research/hy4. Existing news summaries do not provide the parameter count, training data, benchmark results, code repository, or model weights.

The boundaries are clear: the Hy4 research page and its Hacker News listing can be verified. Missing details on the official page cannot be filled in through community speculation. The goal here is not to rank the model, but to examine the questions it raises and the answers it makes independently verifiable.

Hy4 Reaches the Front Page, but the Official Summary Is Missing Key Details

Developers typically ask three questions first: How large is it? How was it trained? What did it score?

Hy4’s current public summary leaves those questions unanswered:

  • Parameter count, and whether it uses a dense or MoE architecture;
  • Training data composition, dataset size, and data cutoff date;
  • Benchmark results, evaluation setup, and reproduction scripts;
  • Context length and real-world long-context performance;
  • Inference throughput, hardware configuration, and deployment requirements;
  • Model weights, code, and license.

These are not excessive demands.

Two models may both be labeled “70B” yet differ by several times in deployment cost because of differences in training token count, active parameters, sequence length, and inference framework. The same benchmark result can also change with the prompt, few-shot examples, decoding parameters, and sampler.

Hy4’s first release should therefore explain what was actually rebuilt, rather than why the architecture was rebuilt. Without concrete architectural and training evidence, “rebuilding the architecture” can easily become a marketing phrase.

Inside Hy4: The Focus Should Be on Information Flow, Not Parameter Count

The information flow of an LLM can initially be reduced to a simple chain:

input tokens → representation and positional processing → computational layers → output probabilities → decoding → next input

An architectural change usually does more than replace one module; it alters how information moves through the entire chain.

The first area to examine is context handling.

Long-context capability involves at least three issues: whether the model can accept long inputs, whether its attention or state mechanism can retain distant information, and whether it can use earlier content reliably during generation. Listing a maximum context window of 128K, 256K, or 1M does not answer those questions.

The relevant tests include long-document retrieval, multi-hop reasoning, cross-paragraph constraints, and hallucination outside the provided context. A model may be able to process an entire book without reliably locating a specific passage in it.

The second area is the reasoning path.

If Hy4 changes interlayer connections, state updates, attention computation, or information compression, Tencent should publish architectural diagrams, tensor shapes, and parameter allocations. Developers need enough detail to map the claims to an implementation.

The third area is the training objective.

With the same architecture, pretraining, distillation, supervised fine-tuning, and preference optimization can produce very different behavior. Comparing only the final weights, without examining the training path, makes it difficult to determine whether the gains came from the architecture or from more data and longer post-training.

The fourth area is the inference path.

A deeper network may improve capabilities while also increasing memory-access and communication overhead. If Hy4 introduces sparse computation, state updates, layered caching, or other mechanisms, it should at least answer the following:

  • Does computation per token increase or decrease?
  • How does KV cache usage change?
  • Are batching and concurrent throughput affected?
  • Can quality be maintained after quantization?
  • Can it run on vLLM, SGLang, or TensorRT-LLM?

The value of an architectural innovation ultimately comes down to token cost and latency, not parameter count alone.

Beyond Performance, Four Costs Need to Be Calculated

Open-source models are often treated as little more than leaderboard entries. In reality, at least four costs must be considered.

The first is training cost.

Training cost is not just a matter of GPU hours. It also includes data cleaning, failed experiments, parallelization strategy, and checkpoint overhead. MoE can reduce the active computation per token, but it also adds routing, load-balancing, and deployment complexity. Dense models have simpler architectures, but every inference must compute across all parameters.

The second is inference cost.

For production services, comparisons should include throughput, time to first token, latency per token, and concurrency. Long-context workloads also require accounting for the memory and bandwidth cost of the KV cache.

Strong single-user performance does not mean a model remains cost-effective under concurrent load. Offline evaluations cannot replace production stress tests.

The third is deployment complexity.

A weights file is not the same as a usable product. Users need to know:

  • Whether the model can be loaded directly into mainstream inference frameworks;
  • Whether the tokenizer, chat template, and special tokens are complete;
  • Whether quantized versions are available;
  • Whether Docker, vLLM, or SGLang examples are provided;
  • Whether there is a clear partitioning strategy when GPU memory is insufficient.

The fourth is license restrictions.

“Downloadable” is an access condition, not a usage condition. Commercial use, redistribution, derivative models, output use, and liability terms all depend on the specific license.

For that reason, “open source” cannot be determined by whether model files appear on GitHub. The community must be able to download them legally, run them easily, evaluate them independently, and continue developing from them.

Code, Weights, and Reproduction Are Hy4’s Hard Requirements

The community does not need a statement that the model is “open source.” It needs a set of inspectable artifacts:

Artifact What the community needs Problem if it is missing
Model weights Clear version, format, and license The model’s actual behavior cannot be inspected
Inference code A runnable loading example The model cannot be launched after download
Configuration Tokenizer, chat template, and special tokens Results cannot be aligned
Data documentation Sources, scale, cleaning, and mixture ratios Bias and contamination cannot be analyzed
Evaluation scripts Prompts, decoding parameters, hardware, and versions Benchmarks cannot be reproduced
Training code Architectural implementation and training configuration The architectural innovation cannot be verified
Cost data GPU model, duration, throughput, and memory usage Economic efficiency cannot be compared

There is currently no evidence that Tencent has released all of these materials. The accurate statement is that the existing summary does not provide them; whether the official page includes them must be determined from the public materials and repository.

There is also no basis for fabricating a set of Hy4 installation commands. The available source material contains no README or real interface definitions.

# Example purpose: check whether the community provides a reproducible entry point
from pathlib import Path

required = [
    "README.md",
    "config.json",
    "tokenizer.json",
    "evaluation/",
]

root = Path("./hy4")
missing = [item for item in required if not (root / item).exists()]

if missing:
    raise SystemExit(f"Missing reproduction artifacts: {missing}")

print("Repository contains the basic reproduction artifacts.")

This code neither downloads the model nor assumes Tencent has a unified repository. It simply separates the requirements for running the model from those for reproducing it, making them inspectable through a set of files. Once the official interface is defined, it can be replaced with the actual pip install command, model ID, and CLI commands.

If Hy4 ultimately offers only a research page, a demo interface, and a handful of scores, it will amount to a technology announcement. If it also provides code, weights, training documentation, and evaluation scripts, it could become a complete model system.

Editorial Assessment

Hy4 matters not because it reached the Hacker News front page, but because it highlights a common gap in large-model releases: promotional materials can appear quickly while the conditions required for reproduction are routinely left out. Developers need more than a model name and leaderboard positions. They need to download, run, and rerun the model under the same configuration, as well as examine its failure cases.

If Tencent next releases the weights, evaluation scripts, and realistic inference costs, discussion can shift from whether the architecture is novel to whether its capability and cost per token are competitive. That would be more useful to teams choosing a model and more representative of production than another parameter comparison table.

What distinguishes this release from some earlier ones is that Hy4 makes an explicit technical claim: that it has rebuilt the architecture. That makes the missing evidence easier to identify. Without structural diagrams, training configurations, and reproduction materials, the claim remains unverified rather than established. The available public information is insufficient to confirm the specific architecture, parameter count, training data, or license.

Sources

Source: https://hy.tencent.ai/research/hy4