Jensen Huang's next play isn't a faster chip—it's CUDA, DPUs, and the AI Factory stack. Most competitors are aiming at the wrong target.
A recent TechCrunch piece made the point bluntly: Nvidia's AI advantage is moving beyond the GPU. The bigger takeaway isn't that Nvidia keeps winning—it's that most rivals are still fighting last decade's war.
On raw per-card compute, AMD has nearly closed the gap. But Nvidia's moat was never really about silicon.
GPUs Are Already Nvidia's Weakest Layer
Here's an unfashionable take: competing with AMD and Intel on per-card TOPS is the least meaningful race in AI silicon right now.
The AMD MI300X launched in 2023 with memory bandwidth in the same league as the H100 and FP16 throughput that bites hard. Two years later, why hasn't the market shifted?
Open PyTorch's source and look:
# PyTorch default tensor backend
import torch
x = torch.randn(1024, 1024).cuda() # runs without .cuda(), but CUDA is the default
print(torch.backends.cudnn.version()) # cuDNN version
torch.cuda is PyTorch's de facto standard. That cuda is far more than "works on Nvidia hardware." Underneath sit cuDNN, cuBLAS, NCCL, and tens of thousands of hand-tuned operators. Replacing the cuda call means rebuilding the entire deep learning stack from scratch.
MI300 has been compute-competitive for two years. PyTorch's default backend is still CUDA.
CUDA Is the Actual Moat
Hardware can catch up in a couple of product cycles. Software ecosystems take a decade.
Nvidia's CUDA isn't a single product—it's a tightly interlocked set of libraries:
- cuDNN: optimal implementations for convolutions, attention, normalization, and other DL primitives
- TensorRT: inference optimizer (quantization, kernel fusion)
- NCCL: multi-GPU and multi-node collective communication, the backbone of distributed training
- Triton (the GPU version): Nvidia's intermediate kernel language
Where does developer stickiness come from? When you write F.conv2d(x, w) in PyTorch, it silently calls hand-tuned cuDNN assembly matched to the current GPU architecture. That "free" performance disappears the moment you move to ROCm.
Open alternatives exist—Triton, ROCm, Ascend C are all catching up. But running a model demo on your laptop and training stably for a month on 1024 GPUs are completely different games. The former is a weekend project; the latter needs a full SRE team babysitting stability.
DPU + NVLink: Moving the Battlefield to the Network
CUDA locks down the software layer. Nvidia's second moat sits in networking.
What's the real bottleneck on an 8x H200 node? Not compute—the speed at which GPUs talk to each other. NVLink bandwidth jumped from 900 GB/s to 1.8 TB/s between generations. AMD's Infinity Fabric and Intel's Xe Link are still an order of magnitude behind.
Then there's the BlueField DPU. Nvidia crammed the NIC, storage controller, and security engine into a single Arm-core SoC. The practical effect:
- Data travels from storage to GPU without the CPU handling copies
- Collective communication offloads to the DPU; the main GPU just computes
- Multi-tenant isolation and hardware-level encryption
Intel and AMD have no real counter here. Intel's IPU and AMD's Pensando are still catching up, measured in years.
AI Factory: Nvidia Doesn't Sell Cards, It Sells Plants
The most underappreciated shift is Nvidia's business model.
Customers used to buy A100s and H100s. Now they buy DGX SuperPODs, Foundry capacity, and NIM microservices. A typical contract looks like this:
# NIM (Nvidia Inference Microservice) deployment sketch
from nvidia.nim import Llama3_8B_Instruct
model = Llama3_8B_Instruct(
gpu="H200-80GB",
replicas=8, # 8-way tensor parallelism
precision="bf16", # bfloat16
max_batch_size=64,
)
response = model.generate("Explain the NVLink protocol")
Customers don't need to know how to initialize NCCL, configure RDMA NICs, or write CUDA kernels. One click and you have a production-ready inference service.
Switching cost moves from "swap a card" (hundreds of thousands of dollars) to "replace the entire SRE pipeline" (tens of millions). That's why customers complain about Nvidia's pricing while quietly signing the checks.
Where Chinese Chipmakers Are Missing the Point
In China's current AI chip wave, Cambricon, Hygon, and Moore Threads are all racing to match FP16/INT8 specs. That's necessary—but nowhere near sufficient.
The real problem: the CUDA ecosystem is almost nonexistent in China.
Developers don't need "can run PyTorch"—CUDA has run PyTorch for a decade. They need "runs just as fast, just as stable, with just as few bugs." That requires:
- Deep integration with upstream PyTorch contributors (not a fork—actual commits into main)
- Official support from major LLMs (Llama, Qwen, DeepSeek)
- Engineering muscle to run a cluster at full load for 30 days with zero crashes
Hardware parity is hard. Ecosystem parity is harder. That gap is the deepest moat Chinese chipmakers will face over the next three years.
Editor Take
Why does this matter? It redefines what an "AI chip company" moat actually is.
For a decade, the default assumption was "better silicon equals a better moat." Nvidia spent fifteen years proving that the compound advantage across software, networking, and system integration is far harder to dislodge than per-card specs.
My read: over the next three years, Nvidia's most dangerous competitors won't be AMD or Intel. They'll be the combination of hyperscaler in-house silicon plus a cross-vendor software abstraction layer. AWS Trainium 3, TPU v6, and Tenstorrent don't threaten Nvidia's chip business directly—they threaten its "AI Factory" narrative. If customers start accepting a hybrid architecture where inference runs on in-house chips and training stays on Nvidia, those margins will compress.
The real long-term threat to CUDA may also come from cross-vendor abstraction layers like Triton. OpenAI has been using it internally across the board for years, and it's recently opened up to AMD and Intel. If Triton becomes the new "PyTorch backend," Nvidia's cuDNN/NCCL lock-in gets decoupled. It's a slow variable, but the direction is worth tracking.
The uncertainty: Nvidia sees this coming, which is why it's been pushing higher-level lock-in points like NIM and Megatron-Core. The next two years will be about who captures the developer's mental default stack—Triton or NIM. That choice will shape the 2028 landscape.