Opinion

Models Are Eating the Agent Harness — What Survives?

LangChain-style scaffolding is being absorbed into model weights. The next layer of abstraction won't manage context for models — it'll manage attention for humans.

Font size

Core thesis from Latent Space's The Evolution of the Agent Harness:

"Models keep absorbing the harness into their weights — soon, it will be a harness for human attention rather than for the model."

What the Harness Is, and Why It's Only Now Getting Attention

The harness is easy to define. It's the glue wrapped around the model:

  • prompt templates
  • tool calling schemas
  • memory management
  • retrieval
  • retry and fallback logic
  • multi-model routing
  • permissions, approvals, and state persistence

In 2023, it was essential. GPT-3.5 couldn't reliably emit the JSON a function call required. Developers wrote format constraints, exception branches, regex extraction, and retry logic. It was ugly, but it worked.

By the time GPT-4o and Claude 3.5 Sonnet arrived, models started handling tool calls natively — not just generating an answer, but deciding which tool to invoke and which parameters to pass.

Capabilities that lived in the harness started moving into model parameters.

Not an upgrade. A migration of abstraction layers.

Models Are Eating the Harness, One Bite at a Time

When OpenAI shipped function calling in June 2023, developers treated it as an API-level mechanism: the model handled natural language, the application parsed structured calls.

The typical middle layer looked like this:

import json
import openai

def classify_ticket(text: str) -> dict:
    response = openai.chat.completions.create(
        model="gpt-3.5-turbo-0613",
        messages=[
            {
                "role": "system",
                "content": (
                    "Return JSON with keys `category` and `priority`. "
                    "Category must be one of: billing, technical, account. "
                    "Priority must be high, medium, or low."
                ),
            },
            {"role": "user", "content": text},
        ],
    )

    return json.loads(response.choices[0].message.content)

This code stuffed the protocol into a system prompt. The model had to produce valid JSON on its own, and the application dealt with missing fields, invalid enums, and format drift.

By the GPT-4o era, OpenAI had turned tool calling into a native model interface. Applications only needed function declarations and execution:

import json
import openai

client = openai.OpenAI()

tools = [
    {
        "type": "function",
        "function": {
            "name": "classify_ticket",
            "description": "Classify a customer support ticket",
            "strict": True,
            "parameters": {
                "type": "object",
                "properties": {
                    "category": {
                        "type": "string",
                        "enum": ["billing", "technical", "account"],
                    },
                    "priority": {
                        "type": "string",
                        "enum": ["high", "medium", "low"],
                    },
                },
                "required": ["category", "priority"],
                "additionalProperties": False,
            },
        },
    }
]

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "I was charged twice this month"}
    ],
    tools=tools,
)

arguments = json.loads(
    response.choices[0].message.tool_calls[0].function.arguments
)
print(arguments)

Format constraints didn't disappear. They moved from prompts into schemas, then got consumed by the model's native tool calling interface.

Anthropic followed the same path. Claude 3.5 Sonnet's tool use reached production-grade reliability. Each new layer of native capability stripped away a layer of external harness.

Llama 3's improved instruction following did the same thing. Few-shot examples used to demonstrate task format. Now, structured output, function calling, and long-instruction following get trained into the model directly.

LangChain's trajectory is the most visible footnote to this shift.

It grew fast in 2023 because agents lacked tools, memory, and chaining — developers genuinely needed a unified interface. By 2024, the community kept circling back to "is LangChain over-engineered?" GitHub star growth flattened. "LangChain winter" became an in-joke.

The criticism isn't just about clunky APIs. The deeper issue: models are taking over coordination work that frameworks used to handle.

The Next Harness Serves a Different Customer: Humans, Not Models

The sharpest insight from Latent Space isn't "LangChain will disappear." It's the role reversal.

The next generation of harness may not primarily serve the model. It serves human attention.

Models now produce 20 pages of code, a 50-item plan, or a 10,000-word analysis in a single output. Traditional UIs just dump all of it into a chat window. The scrollbar keeps moving. Users hunt for differences, risks, and next steps on their own.

But what humans are actually short on isn't context window — it's attention.

The new interface needs to tell users, while the model is still running:

  • which lines of code changed
  • what's still a draft
  • which action needs approval
  • where unresolved risks are hiding
  • what to look at next
  • what's safe to ignore

Latent Space summarizes this as "a harness for human attention," not "a harness for the model."

The distinction matters.

A model-facing framework optimizes for token consumption, tool selection, and retry counts.

A human-facing interface optimizes for cognitive load, information density, and decision pacing.

The former can be absorbed through training data and model capability. The latter requires understanding what the human is actually doing.

How Much Moat Is Left in the Orchestrator Space?

If LangChain, LlamaIndex, CrewAI, and AutoGen are only "making prompts look prettier," their moats are effectively zero.

Worth breaking this down.

Orchestration Isn't Dying

Complex agents still need state, conditional branching, concurrency, timeouts, persistence, and recovery. Real-world workflows don't vanish just because models get smarter.

But "can connect LLMs to tools" is becoming worthless. Model APIs already ship structured output and native tool use. SDKs keep absorbing common design patterns.

The Real Scarcity Is the Feedback Loop

What survives looks more like:

  • evaluation: running real task suites to measure success rate, cost, and latency
  • observability: logging every prompt, tool call, state change, and failure reason
  • human-in-the-loop: inserting human approval at critical nodes instead of letting the model run end-to-end
  • permissions and security: controlling what the agent can read and write
  • state recovery: resuming from the right point after a process crash

These capabilities sit close to the model but haven't been absorbed into model weights. They depend on external systems, real users, and organizational workflows. The bigger the model gets, the more developers need to know when it fails — not just how to write another while True.

That's why stateful runtimes like LangGraph still have room. What they sell isn't "can call an LLM" — it's turning state transitions and recovery into an inspectable engineering system.

The moat isn't the number of abstractions. It's observability.

Where Engineers Should Bet

Three suggestions.

1. Stop Writing New Orchestrator Abstractions

Don't build another framework just because you can write a state graph, tool decorator, or prompt chain.

Start with a more concrete question: what's missing from existing models, SDKs, and protocols?

If the answer is "wrap LLM calls in a unified interface," that value rarely survives a single model upgrade cycle.

2. Redirect Effort Toward Evaluation and Observability

Every agent project needs a task suite, not just a demo.

At minimum, log:

from dataclasses import asdict, dataclass, field
from time import perf_counter

@dataclass
class Trace:
    task_id: str
    model: str
    input_tokens: int = 0
    output_tokens: int = 0
    tool_calls: int = 0
    latency_ms: int = 0
    success: bool = False
    tags: list[str] = field(default_factory=list)

def trace_agent_run(task_id: str, run) -> Trace:
    started = perf_counter()
    result = run()

    return Trace(
        task_id=task_id,
        model=result.model,
        input_tokens=result.input_tokens,
        output_tokens=result.output_tokens,
        tool_calls=len(result.tool_calls),
        latency_ms=round((perf_counter() - started) * 1000),
        success=result.exit_code == 0,
        tags=result.tags,
    )

Real benchmarks beat framework landing pages.

On the same task suite, compare model versions, prompts, tool definitions, and retry strategies. Failed cases should enter regression tests. Otherwise the agent "succeeds in demos" and relies on human firefighters in production.

3. Study Human Attention Management

Product patterns are already emerging.

OpenAI launched ChatGPT Canvas in October 2024. It doesn't just display text — Canvas can highlight sections that need changes, support direct editing, and process specific regions independently. Users don't have to hunt through chat history for the right version.

Anthropic launched Claude Artifacts in June 2024. After generating code, documents, or small web pages, Artifacts moves the output into a dedicated workspace. Users can view, edit, and iterate alongside the model conversation.

Both solve the same problem: don't stuff a 10,000-word answer into a scrollable window.

This is where the next wave of agent products should compete:

  • auto-collapsing repeated content
  • flagging sections with uncertain facts
  • putting diffs, evidence, and execution results back in their original context
  • approving risky operations
  • pushing only key status updates during long tasks
  • letting users edit from any point instead of regenerating everything

So Who Wears the Remaining Skin?

The harness won't disappear. It will get eaten by models one layer at a time.

Prompt templates get eaten by instruction following. JSON extraction gets eaten by structured output. Tool selection gets eaten by native tool use. Short-term memory gets eaten by long context.

What's left isn't another more complex agent class. It's two external systems:

  • On the machine side: state, permissions, evaluation, and observability.
  • On the human side: attention, comprehension, and decisions.

The first decides whether agents can run reliably. The second decides whether humans will actually use them.

The stronger models get, the less time developers should spend re-wrapping prompts.

Time to fill the holes the harness leaves behind.

Reference: The Evolution of the Agent Harness — Latent Space

Source: https://www.latent.space/p/attention-interface