LLMs

Microsoft's Decision-1 Bets on Small and Fast

Microsoft quietly shipped Decision-1, a dedicated decision-making model on Azure AI Foundry. The agent stack's middle layer might be more important than anyone realizes.

Font size

A fairly quiet post recently landed on the Hacker News front page: microsoft-decision-1-model-foundry, with 124 points and 47 comments. The publisher is Microsoft's Command Line engineering blog, and the model lives on Azure AI Foundry.

The post itself got middling traction, but the product positioning is the interesting part: this is a dedicated decision-making model, not a chat model. Microsoft's summary states it plainly: "our model for fast decision-making."

The decision-mocking tier of the agent stack has been neglected for a long time. I went through everything available and here's my take.

What Decision-1 actually is

Decision-1 is not built to chat with you. Its task surface is narrow: given a limited time budget, pick the next action from an action space.

In agent terms, it handles the "Thought → Action" beat inside a ReAct loop:

  • Should a browser agent click this button?
  • What should a customer-support agent ask next, and which API should it hit?
  • Which tool should a workflow agent pick among several options?

How does this differ from a general-purpose LLM? Speed. Not the "deeper reasoning" kind of fast, but literal low latency and low per-step cost.

Microsoft has not published parameter counts or benchmark numbers (more on that later). From the Azure AI Foundry positioning and the name (Decision-1, with the "1"), this looks like the first entry in a series. The goal is proving the direction works, not claiming SOTA.

Note: Command Line is an engineering content channel Microsoft started using last year. It's developer-focused and sits apart from the traditional Azure docs and marketing blogs.

Why build a dedicated decision model

This is where it gets interesting.

General-purpose models can already do "everything." Why ship a dedicated decision model? The contrarian read: what actually bottlenecks agents in production is rarely capability; it's speed and cost.

Consider a RAG agent running a complex task. It might loop 10 to 20 times. If every step uses GPT-4o or Claude 3.5 for the decision, the decision latency alone stacks up to several seconds, sometimes over ten. Token costs get painful as well.

Swap in a small decision model:

  • Per-step latency drops from 800ms to 100-200ms
  • Per-step cost drops from a few cents to fractions of a cent
  • Over a 20-step loop, total latency drops from 15 seconds to 3. The user experience is completely different.

The core value of a dedicated decision model is escaping the cost and latency trap of general-purpose LLMs. It doesn't need to write poetry or hold open-ended conversations. It just needs to pick correctly and quickly inside a constrained action space.

The analogy is database indexes. You have the full table (the general LLM) and you have the index (the dedicated decision model). Most queries hit the index just fine; only complex ones require the full table scan.

Position within the Microsoft Foundry ecosystem

The post is hosted on commandline.microsoft.com, Microsoft's engineering blog domain, but the URL itself includes "model-foundry," which signals that the model is tied to the Azure AI Foundry platform.

What this implies:

  1. Not a standalone API sale. It's part of the Foundry ecosystem. Enterprise users will essentially need to run on Azure.
  2. Deep integration with Microsoft's agent frameworks. Microsoft has been pushing Microsoft Agent Framework (the evolution of AutoGen), and Decision-1 is likely a default decision-maker option within that stack.
  3. Enterprise market orientation. Microsoft sells agent solutions primarily to enterprises, which care about latency, cost, and compliance — exactly the strengths of dedicated models.

Microsoft's strategy is becoming clearer: rather than going head-to-head with OpenAI and Anthropic on "biggest and strongest" general models, the company is building out the middle layer for enterprise deployment. Foundry is the platform, Decision-1 is the dedicated component, and Agent Framework is the orchestration.

Comparison with similar approaches

Several categories of work are tackling similar problems.

1. Anthropic's "fast model" track

Claude 3.5 Haiku fits this positioning: small, fast, cheap. But Haiku is still a general-purpose chat model with a smaller parameter count. Decision-1 is task-specialized, with a fundamentally different training objective.

On execution: Haiku running a decision task still has to interpret context first, then generate a decision. Decision-1 assumes the context is already structured and outputs the action directly.

2. OpenAI's o-series reasoning models

o1 and o3 go in the opposite direction: more compute for deeper reasoning. Decision-1 goes the other way: less compute for faster decisions.

The two don't conflict. The use cases differ:

  • Complex planning, one-shot decisions → o-series
  • High-frequency loops, per-step decisions → Decision-1

3. RL decision small models (robotics and games)

Academia has done this work for years: Decision Transformer, Trajectory Transformer, and various robotics RL policies. Most of it lives in simulation, though. Production agent stacks don't see much of it.

Decision-1's differentiation: designed for production agent workflows, not games or robots. Inputs are tool/API schemas and current state. Outputs are the next action call, not continuous control signals.

4. Open-source small-model track

Qwen2.5, Llama 3.1, and similar open-source small models can be fine-tuned into capable decision-makers. But enterprises rarely fine-tune themselves — data is scarce, engineering cost is high. Microsoft offering a hosted, ready-to-use decision model is a modest value for small companies and a substantial one for enterprises.

ApproachPositioningStrengthsWeaknesses
Claude 3.5 HaikuSmall general modelVersatile, mature ecosystemNot decision-specialized; latency still has room
OpenAI o-seriesDeep reasoningStrong on complex decisionsSlow and expensive; not suited to looped calls
Open-source small model + fine-tuningSelf-built decision-makerControllable and customizableHigh engineering cost; data required
Decision-1Task-specializedFast, cheap, enterprise-grade hostingClosed source; ecosystem still early

Editorial thoughts

What makes this genuinely interesting is that the middle layer of the agent stack is underappreciated.

Industry attention is mostly pulled to two ends:

  • One end: "bigger general models" — GPT-5, Claude 4, Gemini 2, and so on
  • The other end: "more complex agent frameworks" — LangGraph, AutoGen, CrewAI, and other orchestration layers

The decision-model layer in between is something almost no one is doing. Everyone defaults to general-purpose models filling that role.

Decision-1's logic: for some things, using a general LLM is too expensive. It's like using MySQL where you need Redis. Both can store data, but the fit is wrong and the efficiency differs by orders of magnitude.

My bet: over the next 1-2 years, more "specialized middle-layer" models will appear. Decision-making is one, but it won't be the last. Possible additions:

  • Dedicated reranker models (for RAG)
  • Dedicated tool-selection models (for agents picking tools)
  • Dedicated guardrail models (for output compliance checks)

Microsoft has taken a step, but whether it pays off is uncertain. The biggest risk is being absorbed by the general models. If GPT-6 or Claude 5 pushes per-step decision latency under 100ms and prices it near dedicated models, the case for Decision-1 weakens considerably.

Another uncertainty: Microsoft hasn't released benchmark numbers or parameter counts. They might still be iterating; the numbers might also not be flattering. HN engagement was middling (124 points is moderate for an engineering post). The community is watching.

Regardless, tearing the decision model out of the general model feels like the right direction. What matters for agents in production isn't how clever a single step is; it's how smoothly the overall loop runs.

Source: https://commandline.microsoft.com/microsoft-decision-1-model-foundry/