Opinion

Claude Code Verification: Less Code, More Evidence

Anthropic’s engineering team explains how it integrates automated tests, static analysis, and human judgment into the AI coding workflow, with practical guidance for building a CI verification gate.

Font size

Claude Code puts verification before generation. This layered approach determines whether an AI Agent can take ownership of real-world projects.

Claude Code Verification: Less Code, More Evidence

Anthropic has published “How the Claude Code team thinks about verification,” outlining the Claude Code team’s approach to validating code.

The article comes from Anthropic’s official engineering team. Its focus is not a particular feature, but how AI coding agents can use evidence to control what they generate.

The key change is not that the model has become better at writing code. It is that verification now influences the generation process itself.

The link reached the Hacker News front page, earning 119 points and 123 comments. How a team verifies code internally directly affects whether an AI Agent can take responsibility for real projects.

Verification Is Not a Post-Generation Patch

A traditional development workflow usually places testing after coding:

  1. A developer implements a feature.
  2. They write or extend the tests.
  3. They run the tests and fix any failures.
  4. They submit the change for code review.

With AI coding agents, this sequence can take a different form. An Agent generates a large body of code, then repeatedly patches it based on test results.

This may appear faster, but it concentrates risk in the later stages. The more code that is produced, the more edge cases may be missed. If newly generated code “explains away” a bug, the result can even be a false sense of internal consistency.

The Claude Code team brings verification into the workflow earlier. Before writing an implementation, the agent establishes how the requirements translate into checkable conditions.

For example, suppose an endpoint must enforce two input constraints, and its return value must remain compatible with existing callers. The verification task should not consist solely of running pytest at the end. It should begin with executable assertions:

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel

app = FastAPI()

class Item(BaseModel):
    name: str
    quantity: int

@app.post("/items")
def create_item(item: Item) -> dict:
    if not item.name.strip():
        raise HTTPException(status_code=422, detail="name must not be empty")
    if item.quantity <= 0:
        raise HTTPException(status_code=422, detail="quantity must be positive")
    return {"name": item.name.strip(), "quantity": item.quantity}

The important part is not FastAPI. It is that the testing intent is first expressed as explicit rules, and the implementation is then held accountable to those rules.

Code that runs proves only that the syntax is valid and some paths work. Accurate delivery requires more evidence.

Implementing Layered Verification

The official article describes a layered inspection process rather than a single test command. Verification can be divided into three categories.

Layer One: Automated Tests

Tests handle repeatable mechanical checks, including inputs and outputs, state changes, failure paths, and regression behavior.

A minimal unit test might look like this:

from fastapi.testclient import TestClient
from app import app

client = TestClient(app)

def test_create_item_rejects_empty_name():
    response = client.post(
        "/items",
        json={"name": "   ", "quantity": 1},
    )
    assert response.status_code == 422

def test_create_item_rejects_non_positive_quantity():
    response = client.post(
        "/items",
        json={"name": "keyboard", "quantity": 0},
    )
    assert response.status_code == 422

This kind of evidence works best in CI. Run it once before a commit and again before a merge. Failure messages must point directly to the constraint that was violated.

If an agent merely says it has “checked the code” without command output, test results, or a failure record, that is not verification.

Layer Two: Static Rules

Static checks catch issues that tests may not cover, such as formatting, types, unused variables, and architectural boundaries defined by project conventions.

A Python project can combine these checks as follows:

ruff check .
ruff format --check .
mypy app
pytest -q

These tools do not understand every aspect of business semantics, nor do they need to. A verification system should assign different classes of problems to different tools.

Tests answer whether behavior is correct. Linters answer whether the code follows formal rules. Type checkers answer whether type commitments remain consistent. Combining all of that into a vague request such as “make sure the code has no problems” leaves the Agent without stable feedback.

Layer Three: Human Judgment

Human judgment covers what machines cannot yet prove: whether the product intent makes sense, whether an API is well designed, whether a change exceeds the requirements, and whether the code is worth maintaining over time.

The Claude Code team’s first-hand experience matters here. AI Agents can accelerate local implementation, but conflicting goals, implicit constraints, and architectural trade-offs in complex projects still require engineering judgment.

Human review should therefore not mean reading through every function once more. A better place to insert it is at the decision points along the change chain: requirement boundaries, interface design, critical behavior, and the final diff.

From Model Self-Checks to Engineering Constraints

Many Agent tools ask the model to “check its own code” before responding. This is model self-review. It can help, but it provides a lower level of evidence.

The reason is straightforward: the model acts as both implementer and reviewer. It may overlook an error it has just made, or describe something that merely looks plausible as “verified.”

A more reliable approach is to turn judgments into engineering constraints:

  • Express requirements as acceptance criteria.
  • Turn acceptance criteria into tests.
  • Assign project conventions to static tools.
  • Require human confirmation for high-risk paths.
  • Attach reproducible commands and results to every conclusion.

A CI verification entry point can remain small:

name: verify

on:
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -r requirements-dev.txt
      - run: ruff check .
      - run: ruff format --check .
      - run: mypy app
      - run: pytest -q

For AI Agents such as Claude Code, tool output matters more than a natural-language self-assessment. When a check fails, the test should provide the exact input, actual output, and expected output. The Agent then has a stable basis for making fixes.

This is the essence of a verification workflow: rather than asking the model to “try harder,” give it cheaper and more consistent feedback.

How This Differs from Traditional Development

Working with Claude Code is not simply a matter of replacing the keyboard with a chat window.

A traditional workflow usually advances one function at a time: implement it, add tests, and fix bugs. Agentic development may modify a dozen files in one pass, touching configuration, dependencies, migration scripts, and callers.

The verification target consequently expands from a single function to the entire change chain:

  1. Does the change cover the original requirement?
  2. Does the new interface remain compatible?
  3. Can the data migration run repeatedly without adverse effects?
  4. Do failure paths roll back or exit safely?
  5. Do the documentation and actual behavior agree?
  6. Do all automated checks pass?

Traditional code review often focuses on the diff. Collaboration with an AI Agent also requires an evidence chain. If the code claims to fix a concurrency issue, for example, it should include a concurrency test or a repeatable reproduction script rather than only a plausible-looking locking change.

The cost of verification changes as well. In the past, testing was often treated as the final cost of coding. In an Agent-driven workflow, it is closer to a runtime control. The more specific the tests, the less the agent has to guess in an unknown space.

Community Debate and the Limits of the Approach

The article reached the Hacker News front page with 119 points and 123 comments. The figures come from the corresponding Hacker News discussion.

The attention raises a reasonable concern: layered verification can prove that rules are followed, but not necessarily that the rules themselves were chosen correctly.

When tests and specifications share the same source, an Agent may encode a mistaken interpretation in the implementation and then write tests that faithfully reproduce that error. Static rules can check types without knowing whether a product decision is sound. Human review is also constrained by time and attention.

Complex projects face three additional practical problems:

  • Legacy tests may be incomplete or may encode old bugs.
  • End-to-end environments are expensive, and an agent cannot fully reproduce them after every change.
  • Verification scripts need maintenance, and too many rules can increase feedback noise.

Verification should therefore not become a numbers contest. More tests are not necessarily better; critical behavior needs sufficient evidence. Payment logic, permission boundaries, and data migrations usually require stricter checks. Display-only copy may not justify a complex workflow.

Claude Code should also be evaluated within these boundaries. It is good at executing scoped changes, but generation speed alone does not establish project reliability. The more meaningful comparisons are whether failures can be reproduced, errors can be located, fixes can be validated automatically through regression tests, and people can review the key decisions within a limited amount of time.

Implementation can begin with three steps:

  1. Write executable acceptance criteria for critical requirements.
  2. Standardize test, lint, and type-checking commands in CI.
  3. Require the Agent to provide failure reproductions, actual outputs, and verification results.

Original sources:

Editorial Perspective

The significance of the Claude Code team’s article lies not in another collection of testing techniques, but in its shift of the AI coding benchmark from “how much code is generated” to “how much verifiable evidence is submitted.” Code generation used to be expensive, so verification was often squeezed. Once Agents make generation nearly free, scarcity moves to acceptance criteria, test design, and failure diagnosis.

Verification capability is unlikely to replace code-generation capability, but it will change how that capability is valued. An agent that quickly produces 2,000 lines of code but cannot explain how its critical behavior is verified may deliver less value than one that changes only 30 lines and includes regression tests.

What distinguishes this approach is that the official engineering team places verification in the first half of the Agent workflow. Testing is no longer cleanup performed after generation; it is a control layer that constrains the model’s actions. Whether this layered method scales linearly across large legacy systems remains uncertain. Rule noise, test-maintenance costs, and bottlenecks in human judgment may still limit its effectiveness. When evaluating tools such as Claude Code, failure evidence and the change chain offer more insight than the apparent completion speed of a demo.

Source: https://claude.dev/blog/how-the-claude-code-team-thinks-about-verification/