Products

Google TPU 8i Max vs. 8tx: Which Fits?

TPU 8i Max targets large-scale inference, while 8tx focuses on training and high-bandwidth cluster networking. Here is how latency, cost, throughput, and scaling shape the choice.

Font size

Google has released two new AI accelerators: TPU 8i Max for large-scale AI inference and TPU 8tx for large-scale training, supercomputing, and high-bandwidth, low-latency networking.

See the official announcement: TPU 8tx and TPU 8i Max.

The story reached the Hacker News front page and received 203 points and 48 comments. That attention does not settle the buying decision. The key announcement is Google’s division of training and inference into two separate infrastructure paths.

Not Just Two Bigger TPUs

Comparing 8i Max and 8tx by peak compute alone can lead to the wrong question.

AI accelerators do not operate as independent components. Once deployed, they also rely on HBM, host memory, networking, compilers, runtimes, and schedulers. Whether a model can saturate the chip, whether compute units remain busy, and whether communication becomes a bottleneck all have a greater effect on the final bill.

The goal of 8i Max is AI inference. Google’s announcement focuses on inference cost and deployment efficiency. It is closer to a production cloud service: requests have latency targets, traffic rises and falls, and models must handle concurrent requests.

8tx addresses a different set of problems: training very large models and running supercomputing workloads. Its defining feature is high-bandwidth, low-latency networking. During training, thousands of accelerators need to synchronize gradients, fetch parameters, and exchange intermediate results. Even fast chips cannot deliver high cluster utilization if nodes spend too long waiting for one another.

A simplified model illustrates the point:

# 粗略估算:分别计算计算时间与通信时间
def training_step_time(compute_seconds, transfer_seconds, overlap=0.0):
    overlap = max(0.0, min(overlap, 1.0))
    return compute_seconds + transfer_seconds * (1.0 - overlap)

fast_chip_but_slow_network = training_step_time(0.8, 2.4, 0.2)
slower_chip_with_faster_network = training_step_time(1.0, 0.6, 0.7)

print(fast_chip_but_slow_network)       # 2.72
print(slower_chip_with_faster_network)  # 1.18

This is not a performance test of 8i Max or 8tx. It shows why single-chip metrics are insufficient. By making networking central to the positioning of 8tx, Google signals that cluster communication now matters as much as compute itself.

8i Max Is Designed Around Inference Economics

Buying for inference follows different economics from buying for training.

Training is often a fixed project whose scale is chosen after the model is defined. Inference can run indefinitely across many use cases, including search, recommendations, customer service, content generation, and Agent backends.

For organizations buying the platform that includes 8i Max, four measures usually matter most:

  • Cost per token or per million tokens;
  • Sustained throughput rather than a laboratory peak;
  • Tail latency under long-tail request patterns;
  • Whether existing GPU capacity can be absorbed at high utilization.

Google positions 8i Max as a product for large-scale AI inference, emphasizing inference cost and deployment efficiency. The offer is not merely a “faster board”; it is a lower cost per unit of service.

One caveat remains: an official announcement is not an independent benchmark. Results can vary widely with the model, batch size, input and output lengths, and quantization precision. Tool calls in Agent workloads, long contexts, KV cache behavior, and concurrency patterns can also change the cost structure.

Before purchasing, I would replay the organization’s own traffic rather than rank the products by peak FLOPS. At minimum, record:

  1. End-to-end throughput;
  2. P50, P95, and P99 latency;
  3. Power consumption and cost at the target throughput;
  4. Time required for model upgrades, scaling, and failover migration.

If 8i Max can reduce the bill for an existing inference service without materially increasing latency, it has clear value. If the advantage is only higher chip specifications without improvements in compilation, batching, and cluster scheduling, the software stack may erase it.

8tx Competes on Cluster Networking

8tx has a more focused goal: large-scale training and supercomputing.

These customers are not primarily concerned with how quickly a single request returns. They care whether thousands or tens of thousands of accelerators can operate as one system. Common bottlenecks include intra-node bandwidth, inter-node networking, synchronization strategy, checkpoint writes, and recovery time after failures.

Large-model training has its own rhythm. Computation is often not the most expensive phase in the early stages; data loading, expert-parallel communication, and checkpointing can all leave expensive hardware idle. High-bandwidth, low-latency networking reduces the time accelerators spend waiting for data.

This is why a simple “which one is stronger?” comparison between 8tx and 8i Max does not work. The former optimizes aggregate cluster throughput and parallel training efficiency, while the latter optimizes the cost of online inference. Different objectives produce different winners.

For model teams evaluating 8tx, the relevant comparison covers the full training cycle:

  • Samples per second for the same model and batch size;
  • Linear efficiency after scaling to more nodes;
  • Separate tests for communication-heavy and compute-heavy models;
  • Maturity of checkpointing, recovery, and monitoring tools;
  • Reusability of configurations when moving from a test cluster to production.

Until independent benchmarks are available, training speed should not be inferred from the chip name, nor should any specific reduction in training cost be assumed.

Google Is Redrawing Cloud Product Boundaries

Google Cloud is not simply adding two TPUs alongside general-purpose GPUs. It is reorganizing the boundaries between its services.

Specialized TPUs and general-purpose GPUs will coexist. The GPU ecosystem is mature, with broad model compatibility and a more complete set of software tools, making it well suited to rapid experimentation, mixed-model deployments, and specialized workloads. TPUs combine chips, networking, compilers, and cloud services to gain cost advantages for particular workloads.

This differs from selling raw compute. Customers ultimately purchase a complete service, not a chip SKU:

应用
  ↓
模型与推理框架
  ↓
编译器、运行时、调度器
  ↓
TPU 集群与高速网络
  ↓
存储、监控、权限与运维

Google is well positioned to package TPUs as a complete AI infrastructure service. For teams already using Google Cloud, this integration may reduce data movement and system assembly. Research teams that purchase hardware solely for training must still weigh price, availability, and the barriers to migration.

The separation between 8i Max and 8tx also resembles a coordinated product portfolio: online inference and offline training are optimized independently rather than explained through a single hardware metric.

Developer Interest Is Not a Purchasing Decision

The announcement reached the Hacker News front page and earned 203 points and 48 comments. That level of discussion shows that the new TPUs have reached developers, but the concerns raised in the comments are separate from an enterprise decision to migrate workloads.

Developers typically ask:

  • Which frameworks and numeric formats are supported;
  • Whether debugging and profiling workflows are effective;
  • How much existing CUDA code must change during migration;
  • Whether vLLM, Triton, or custom operators are compatible;
  • What real-world throughput and per-token cost look like.

Enterprise buyers have another set of questions:

  • Whether sufficient capacity is available on demand;
  • Whether SLAs and quotas are dependable;
  • Whether failure recovery and technical support are reliable;
  • Whether three-year total cost includes migration and operations;
  • Whether vendor lock-in could cost more than the hardware itself.

The Hacker News score of 203 shows that people care; it does not show that the products are ready to buy. Pricing, regional availability, independent benchmarks, and data from real workloads should not be guessed.

Editorial Take

The significance of this launch is not that Google has added two more chip specifications. It is that competition in AI infrastructure is shifting from individual accelerators to the cost of an entire cluster. Chips, HBM, networking, compilers, scheduling, and operations are being delivered as one integrated system. 8i Max targets the recurring cost of inference, while 8tx targets communication and idle time during training. Each serves a different budget.

Unlike previous TPU launches, Google has more clearly separated training and inference into two product strategies. This does not mean enterprises must choose one or the other. It is more likely that training and online serving will use different hardware at different stages of the same model lifecycle.

My view is that the largest influence on migration cost will not be a 20 percent difference in peak compute. The decisive factors will be whether software adaptation can finish within two weeks and whether the scheduler can maintain high resource utilization. Official positioning, however, cannot replace independent evidence. Pricing, third-party benchmarks, regional availability, and tail latency under real Agent workloads must still be evaluated.

Source

https://blog.google/innovation-and-ai/infrastructure-and-cloud/global-network/tpu-8i-max-and-8tx/

Source: https://blog.google/innovation-and-ai/infrastructure-and-cloud/global-network/tpu-8i-max-and-8tx/