Last week, a post by Boyd Kane titled LLMs could control their host machines by exploiting inference engines (link) landed on the Hacker News front page. Within hours it picked up 19 points and 6 comments — solid traction for a security-themed piece.
This isn't an academic paper or a vulnerability disclosure (no CVE, no PoC repo). It reads more like a roadmap: it lays out overlooked attack surfaces inside inference engines and warns anyone running models locally that the model is not just a statistical function — it can reach back and act on your machine.
The story: a blog post that broke through on HN
Kane's blog lives at boydkane.com, and the post itself is observational rather than theoretical. He notes up front, "this isn't a 0day, it's a chain of known risks."
The HN thread split into two camps:
- One side argued "isn't this just prompt injection rebranded?" Kane pushed back: classic prompt injection tricks the model into speaking. This attack tricks the model into acting, with the execution environment being the host process.
- The other side came from ops engineers listing real incidents they've witnessed: models returning GPU kernel code that the inference engine fed straight to the driver with no sandboxing, or model output getting spliced into deployment scripts as shell command fragments.
Everyone argued, but nobody disputed one thing: no major inference engine currently treats model output as untrusted input.
What the attack surface actually looks like: where vLLM, TGI, and llama.cpp can be breached
The specific paths Kane calls out aren't hypothetical — they're capabilities these engines already expose.
1. Custom operators and fused kernels
vLLM supports user-registered custom CUDA kernels, and TGI's custom_kernels config option loads third-party compiled artifacts. These operators exist for performance tuning. But once model output can influence which operator gets loaded, the attack surface opens. Drop a malicious PTX or SPIR-V payload in, and the GPU driver stack may not catch it.
2. JIT compilation paths
llama.cpp ships with TorchScript-style dynamic compilation on certain branches to speed up inference. If a model's returned "config" or "prompt fragment" can nudge the engine into JIT mode, attacker-supplied code executes inline within the process.
3. Exposed HTTP/RPC ports
Most inference engines bind to 0.0.0.0 instead of 127.0.0.1 by default. I hit this personally while deploying DeepSeek-7B — vLLM's --host 0.0.0.0 is the default. Running locally is fine, but the moment you put it on a cloud box, the entire inference service is exposed to the public internet. Kane's point goes further than misconfiguration though: even when the port is bound to localhost, model output can still affect the engine's internal scheduling logic.
4. Model weight parsing paths
How much fuzz testing has been done on GGUF, SafeTensors, and HuggingFace checkpoint parsers? Far less than on browser engines. Weight poisoning is a known risk (see PyTorch's CVE-2022-29248 from 2022), but GGUF's fuzz coverage is essentially zero.
The core chain looks like this:
user input -> LLM inference -> model output -> engine treats it as "instruction" -> host process behavior triggers
That last step is the crux — model output being parsed, scheduled, or JIT-compiled as if it were trusted input.
Why this isn't the same old problem: a new front beyond prompt injection
Traditional prompt injection targets what the model says: get it to spit out phishing copy, bypass safety guardrails, leak the system prompt. The fallout is informational — text leaks, you delete it.
The attacks Kane describes target what the model does:
- Traditional: the model outputs "please send the password to attacker@evil.com"
- New attack: the model output triggers the engine to load a malicious kernel, giving the attacker a shell on the host
The first is social engineering. The second is RCE. The threat tiers don't compare.
There's a subtler problem too. With classic prompt injection, you can catch it by reviewing model output. In this new attack, the model output can look completely normal — a plain JSON blob, a routine token sequence — yet still trigger a dangerous path inside the engine's parser. All the logs will show is a single line: "loaded custom op."
Local deployment is actually riskier
This was the most upvoted pushback in the comments, and the most counterintuitive finding.
Many individual developers assume: "I run Ollama on my Mac, nothing leaves my machine, so it has to be safer than calling the OpenAI API." Kane's post says bluntly: that assumption is wrong as of 2025.
The reasoning is simple:
| Deployment | Exposure |
|---|---|
| OpenAI / Anthropic API | Network layer, API key management |
| Local Ollama / vLLM | GPU driver, weight parser, CUDA toolkit, system calls |
When you call an API, your attack surface is an API key and network traffic. The weights, parsers, and GPU drivers live on the vendor's side, with dedicated teams hardening them.
With local deployment, all of that runs on your own machine. And the vast majority of individual developers have never fuzzed a GGUF parser or audited llama.cpp's JIT paths. You've taken the security responsibility away from OpenAI's security team and handed it to yourself — but you don't have the capacity to own it.
Worse, Ollama is designed around "just works" defaults, and --host 0.0.0.0 is the norm. Run a Linux workstation with Ollama on your home or office network, and anyone on the LAN can hit your inference port — and behind that port sits a full machine capable of executing arbitrary GPU code.
Why this matters
The genuinely alarming part of Kane's essay isn't the specific vulnerabilities (most have no PoC, and nobody's exploiting them yet). It's the vacuum of security accountability it exposes:
Inference engine authors are mostly AI performance researchers, and their security model still assumes "model output is text." But vLLM and TGI have evolved into complex runtime systems — with schedulers, custom operator registries, dynamic compilation. They're no longer "model inferencers." They're "AI operating systems." Nobody audits them to OS security standards.
Compare this to earlier waves of security discourse:
- 2018's model extraction debate: can the model output harmful content? — an information security question.
- 2023's prompt injection debate: can the model be tricked? — still information-layer.
- Kane's framing: can the model's output become code execution? — this is an infrastructure security question.
My read: within the next 12 months, inference engine security auditing will become its own category. Today, security review for vLLM and TGI leans on community volunteers, and findings surface by accident. The closed API vendors — Anthropic, OpenAI — pour resources into the model layer, but on the inference engine side, especially the open-source side, they're essentially running naked.
What needs to be added, in order of priority:
- A trust boundary between model output and engine internals. Treat model output as untrusted input, at minimum by sandboxing the parser.
- Default-bind to localhost. vLLM, Ollama, and TGI should all default to
--host 127.0.0.1, forcing users to opt in to external exposure. - Fuzz the weight parsers. GGUF and SafeTensors parsing paths need the same continuous fuzzing treatment browsers give to HTML parsing.
I won't promise these attacks will be exploited at scale. Plenty of "theoretically feasible" attacks fade into obscurity. But inference engines have grown complex enough that waiting for the first real PoC to show up before acting will be too late.
References: