Products

Testing the M5 Ultra Mac Studio: Can Local AI Agents Really Replace the Cloud?

Federico Viticci calls the M5 Ultra Mac Studio the dream machine for local AI agents. We look past the unified memory hype to see how it actually handles agent workflows, where it falls short, and whether the math on replacing cloud APIs makes sense.

Font size
⚠️ DRAFT — fact-checking issues below

Federico Viticci at MacStories recently published a review titled The Dream Mac for Local AI Agents. The piece hit the front page of Hacker News last Saturday, peaking at 82 points with 39 comments.

This is not a typical hardware review. Viticci has spent years tracking real-world productivity on macOS and Apple Silicon, and this time he benchmarked the machine through the lens of running actual agents — not video editing, not Xcode builds, but LLM inference with tool calls under load.

Why the Mac Studio drew the spotlight

Apple positions the M5 Ultra Mac Studio for creative professionals, video editors, and music producers — the usual consumer framing.

The Hacker News thread told a different story. Readers cared about:

  • Whether a 70B model can actually run locally
  • How latency compounds across multi-step agent tool calls
  • Whether memory is enough to hold the full context window
  • Whether a workflow can complete fully offline

Mac Studio over Mac Pro? The Mac Pro still ships with Intel Xeon CPUs paired with AMD GPUs, which means it inherits none of the Apple Silicon advantages. The Mac Studio is the strongest Apple Silicon desktop you can buy.

Viticci's review hits a real pain point: cloud APIs are not free, and your data leaves the device. Both issues hit small teams and solo developers hardest.

Unified memory is the real story

The 192GB unified memory configuration on the M5 Ultra was not designed for video editing. Even 8K RAW workflows peak in the tens of gigabytes.

LLM inference is what actually consumes it.

Datacenter GPUs start at 80GB of VRAM — A100, H100, all in that range. Running a 70B model on consumer hardware (roughly 140GB at FP16, around 40GB after INT4 quantization) is effectively impossible on a traditional CPU-plus-discrete-GPU setup. Data has to shuttle across PCIe, which tops out at tens of GB/s and becomes the bottleneck.

Apple Silicon's unified memory architecture sidesteps this entirely. The CPU, GPU, and Neural Engine share a single memory pool with bandwidth in the hundreds of GB/s (the published spec for the M5 Ultra is 819 GB/s).

In practice:

  • Out of 192GB total, the GPU can claim close to 192GB as VRAM
  • A 70B model quantized to INT4 fits entirely in memory
  • No data copying between CPU and GPU

Sounds great, but unified memory is not magic. MLX has matured, and llama.cpp supports the Metal backend, but Apple Silicon GPU throughput is still an order of magnitude behind an NVIDIA H100. Numbers on that below.

Where local agents hurt: the latency stack

Agent UX boils down to three numbers:

  1. Throughput (tokens/s) — generation speed
  2. Time to first token (TTFT) — perceived responsiveness
  3. Compound latency across multi-step tool calls — how much slower a 10-step agent is than a single turn

The third number matters most. An agent is a loop: LLM thinks → calls tool → gets result → thinks again. Each step adds delay.

Community benchmarks (Mac Studio M2 Ultra, MLX backend, llama.cpp quantized builds):

  • Llama 3 70B Q4: roughly 8–12 tokens/s
  • Mistral Large Q4: roughly 6–10 tokens/s
  • TTFT: 300–800ms

The M5 Ultra clocks in roughly 30–50% faster than the M2 Ultra (more CPU/GPU cores, higher memory bandwidth), which puts an optimistic estimate for 70B Q4 at 15–18 tokens/s.

That speed works, but it is not smooth. Conversational interaction is barely tolerable. Run an agent through a chain — search, read a file, hit an API, summarize — and the whole workflow can finish 5–10x slower than the cloud equivalent.

An NVIDIA H100 running the same Q4-quantized model hits 80–150 tokens/s, a full order of magnitude faster. Consumer NVIDIA cards like the 3090 or 4090, with 24GB of VRAM and a mature CUDA ecosystem, also feel noticeably snappier than Apple Silicon — assuming you are willing to run Linux and deal with the setup.

# Minimal example: running local Llama 3 via MLX on a Mac Studio
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Meta-Llama-3-70B-Instruct-4bit")

response = generate(
    model,
    tokenizer,
    prompt="用三句话解释什么是 Agent",
    max_tokens=200,
    temp=0.7
)
print(response)

This is straight from the MLX repo and runs unmodified on a 192GB M5 Ultra (a 70B 4-bit model takes around 40GB).

The real cost of going local

A fully configured M5 Ultra Mac Studio (192GB) runs about $8,000 on Apple's US store before tax.

Cloud API pricing for comparison:

  • GPT-4o: $2.50/M input tokens, $10/M output tokens
  • Claude Sonnet 4.5: $3/M input tokens, $15/M output tokens
  • DeepSeek V3: $0.27/M input tokens (cache hit), $1.10/M output tokens

A typical agent workflow — read five files, hit three APIs, produce a summary report — burns roughly:

  • 50K input tokens (file reads + tool results)
  • 5K output tokens (summary + tool-call decisions)

At Claude Sonnet 4.5 pricing:

Input:  50K × $3/M  = $0.150
Output:  5K × $15/M = $0.075
Per run ≈ $0.225

Run that 50 times a day:

Monthly cost = 50 × 30 × $0.225 ≈ $337

Annual cost ≈ $4,050.

Less than half the price of a maxed-out Mac Studio, and you skip the MLX compatibility headaches, the wait for local inference, and the constant memory ceiling.

On DeepSeek V3 (roughly 10x cheaper than Claude), annual cloud spend drops below $500. At this usage level, the local setup breaks even in about two years — and only if you never touch the cloud. Nobody actually does that, because:

  • Local hardware cannot handle huge context windows (200K+ tokens)
  • Local models still trail GPT-5 and Claude 4.5 on complex reasoning
  • Multimodal (vision, PDF parsing) is significantly worse on local models

My take

This pattern has played out with every "local vs. cloud" inflection point before it.

The M5 Ultra Mac Studio is a transition device, not the endgame. Its real value is not replacing the cloud. It is:

  1. Running privacy-sensitive workloads (medical, legal, internal code) entirely on-device
  2. Handling small-model tasks (7B–13B) that don't need frontier intelligence
  3. Serving as a fallback when offline or disconnected
  4. Giving developers a sandbox to debug local model behavior

Flip it around: the cloud remains the main battleground. An $8,000 box running 70B locally still trails Claude Sonnet 4.5 on quality, runs several times slower, and has roughly half the context window.

The smartest thing about Viticci's review is the title — The Dream Mac for Local AI Agents. He never said "kills the cloud." He said "dream for local." That wording is honest.

My prediction: Apple will keep pushing the unified memory ceiling over the next two years. 192GB is not the end. Once the Mac Pro adopts M5 Ultra silicon, expect 256GB or even 512GB configurations. When local machines can run 400B+ models without bandwidth as the bottleneck, the balance of the conversation starts to shift.

Until then, a hybrid architecture is the pragmatic answer: local for fast, cheap, small-model subtasks; cloud for heavy reasoning, long context, and multimodal. The M5 Ultra Mac Studio is an expensive but very handy piece of that hybrid stack — not a standalone answer.

Sources:

Source: https://www.macstories.net/stories/m5-ultra-mac-studio-review-the-dream-mac-for-local-ai-agents/