Tools

Cloudflare Open-Sources Clef: Decision Models at the Edge

Cloudflare released Clef, a family of small decision models, alongside an RL fine-tuning platform that runs entirely on Workers. The Hacker News thread hit 168 points, with developers debating whether most tasks really need a large LLM.

Font size

Yesterday, Cloudflare dropped something new on its blog: Clef, a class of small models it calls "decision models" — purpose-built for structured decisions rather than chat or text completion. The company also shipped an RL (reinforcement learning) fine-tuning platform that lets you train, deploy, and run inference directly on Workers.

The Hacker News thread hit 168 points and 56 comments by midday. For an infrastructure-focused company like Cloudflare, that level of engagement is unusual — developers clearly have opinions about the idea that not everything needs an LLM.

What Is Clef, and How Does It Differ from an LLM?

Cloudflare's definition is straightforward: decision models are designed for structured decision-making, not open-ended text generation.

In plain terms: feed it a fixed-format input (a feature vector, a user behavior sequence, a set of rules) and it returns a clear decision — a classification label, a routing choice, a risk score, a boolean. It won't write you a poem about autumn.

The differences from a traditional LLM come down to three things:

  • Input/output structure: LLMs take open-domain input and generate freely; decision models take structured input and produce fixed-schema output.
  • Task fit: LLMs are built for "understand + create"; decision models are built for "judge + select."
  • Resource footprint: Decision models are typically orders of magnitude smaller — anywhere from a few MB to a few hundred MB — with millisecond-level latency and dramatically lower cost.

Cloudflare hasn't said what architecture Clef uses, but the positioning suggests something encoder-only or a small transformer optimized for low-latency inference.

Edge Inference Plus RL Fine-Tuning: Cloudflare's Play

Clef isn't a standalone product. It's tightly bound to the broader Workers ecosystem.

The full pipeline works like this:

  1. Use Cloudflare's RL fine-tuning platform to train or adapt a model for your decision task.
  2. Deploy the trained model directly to Workers.
  3. User requests get handled at the nearest edge node, with steady millisecond-level latency.

The appeal isn't "look how strong the model is" — it's how short the path is. The traditional flow looks like: data round-trips to a central cluster → big model runs inference → result comes back. Cloudflare's pitch is: edge node runs inference → result returns. No round-trip, no large model. Latency could drop from 500ms to 20ms; cost from $0.01 per call to $0.0001.

The real play for Cloudflare is platform stickiness. Every time someone builds a fraud-detection system, a smart router, or a content classifier on Clef + Workers, that workload stays on Cloudflare's edge network forever. The model is the bait; the infrastructure is the net.

168 Points on HN: What the Community Is Arguing About

The thread (news.ycombinator.com/item?id=49923692) splits roughly into three camps:

The optimists say this is overdue. One commenter ran the numbers: their company's annual OpenAI API bill could drop by over 80% if they swapped to self-hosted small decision models — with lower latency to boot.

The skeptics see it as marketing. Decision models aren't new, the argument goes — they're what people used to call classifiers. "Any tabular data plus gradient boosting does this. Why involve a neural network?"

The wary crowd worries about lock-in. Once your training, deployment, and inference all live inside Cloudflare's ecosystem, switching costs climb fast. "Free until you depend on it, then the meter starts running."

The skeptics have a point — the concept isn't new. But Cloudflare's value is packaging the entire toolchain onto its edge network. Building this from scratch is a real headache, and "batteries included" goes a long way.

Can Decision Models Replace LLMs? Three Contrarian Observations

My take: no, they can't replace LLMs — but they can take over a lot of tasks that LLMs are wrong for.

Three observations:

1. For classification, routing, and risk scoring, small models often win. LLMs are overkill. Ask one whether a comment is spam and it loads tens of gigabytes of weights, runs tens of billions of floating-point operations, and returns "yes" or "no." A few-megabyte text classifier hits 95% accuracy with two orders of magnitude less latency and three orders of magnitude lower cost.

2. Decision tasks demand predictability far more than creativity. The worst things about LLMs in production are hallucinations, non-reproducibility, and behavioral drift. In fraud detection, a model that says "pass" today and "fail" tomorrow on the same input is an incident. In creative writing, that same "drift" is a feature. Decision models are simpler, more predictable, and easier to keep stable.

3. RL fine-tuning makes specialization feasible. Small models used to lose on two fronts: weak general capability and high customization cost. RL fine-tuning changes the math — you can take a small model and quickly adapt it to your specific task with a modest training set. Cloudflare open-sourcing this toolchain lowers the barrier considerably.

Clef's significance isn't "LLM replacement." It's forcing the question: does every task actually need a large LLM?

Editorial: Why This Round Is Different

Why does this feel different from past "small model vs. big model" debates?

Earlier rounds were mostly about models themselves — how many times smaller Phi-3 is than GPT-4, whether Gemma 2 runs on a phone. Cloudflare isn't dropping a single model; it's shipping model + fine-tuning platform + edge deployment as a complete stack. The leverage of a packaged solution is that it pushes the engineering cost of "using a small model" close to zero.

AI isn't the real goal here. If Cloudflare just wanted to release a model, a HuggingFace repo would do. The actual aim is making Workers the de facto infrastructure standard for AI inference. Every production system built on Clef locks in a long-term customer — same playbook AWS used with S3 free tiers in the early days.

One thing I'm not sure about: how open is the RL fine-tuning platform, really? Does it only support Cloudflare's own models, or any open-source model on HuggingFace? Does it run only on Cloudflare GPUs, or support BYOC? If the answer to both is "Cloudflare only," then "open source" deserves a few asterisks. The blog post didn't go into detail, so this is worth watching when the code repo lands.

Final prediction: over the next 12 months, more companies will peel "tasks that a small decision model can handle" out of their LLM call stacks. This isn't a technology trend — it's an economics trend. When your bill grows 10× year over year and 80% of those calls only need a yes/no answer, rearchitecting is inevitable.

Source: https://blog.cloudflare.com/clef-decision-models/