Most people in the Agent space know Terminal-Bench. Until now, it measured whether an Agent could write code and run commands in a terminal environment—essentially, whether it could handle general programming tasks. The benchmark's creators apparently decided that wasn't enough. "Can it write code" no longer separates Agents; the real dividing line is "can it run a complete research workflow."
That thinking produced Terminal-Bench-Science, a benchmark dedicated to evaluating AI Agents on scientific research workflows. It's not a standalone framework but an extension of the existing Terminal-Bench, shifting the task scope from "write a sorting algorithm" to "reproduce the results of a bioinformatics paper."
It hit the Hacker News front page on launch day, picked up 20 points and four comments. For a new benchmark, that's a respectable debut.
What This Benchmark Actually Measures
First, some context on how Terminal-Bench-Science differs from SWE-bench and GAIA.
SWE-bench tests whether an Agent can look at a GitHub issue, modify code, and fix bugs—the input is an issue description and a repository, the output is a patch. GAIA is broader, testing whether an Agent can handle everyday problems that require multi-step reasoning and tool use.
Terminal-Bench-Science is different. It tests whether an Agent can execute an entire research workflow end-to-end: read the paper → find the data → set up the environment → write the code → run the experiment → handle errors → produce results → interpret them. Any step can break.
The launch page puts it plainly: "We're not testing individual capabilities—we're testing the full workflow." That means tasks aren't designed to score Agents on isolated skills; they're designed to see whether Agents can get things done in real research scenarios.
What the Tasks Look Like: 80+ Real Research Tasks
The benchmark includes 80+ tasks across three domains:
- Biology: gene sequence analysis, protein folding result processing, single-cell sequencing data visualization
- Materials science: crystal structure calculations, first-principles simulation scripting
- Machine learning: paper experiment reproduction, benchmark scoring, dataset cleaning
None of these are "write a function" tasks. Each one requires an Agent to operate continuously for dozens of steps inside a sandboxed terminal. Looking at a few sample tasks, they average somewhere between 20 and 50 calls to shell commands, Python code, and file operations.
The task sourcing is notable—most are extracted from real research scenarios rather than fabricated as benchmark puzzles. That distinction matters. Many Agent benchmarks get criticized for "toy tasks," and Terminal-Bench-Science at least avoids that particular criticism.
One example (from the announcement page):
Task: Starting from a public scRNA-seq dataset, reproduce the UMAP clustering figure from Figure 3 of the paper. The Agent must find the data source on its own, install scanpy, get the pipeline working, handle out-of-memory errors, and produce a comparable figure.
For a human, this is hours of work. For an Agent, it's a serious test.
Current Model Performance: How Far Are We?
The preliminary results on the launch page are sobering.
Mainstream Agents achieve generally low accuracy on the Science subset. I haven't seen the full results table, but based on the announcement's descriptions, scores are well below the 60%+ these same models post on SWE-bench.
GPT-5, Claude 4.5, and Gemini 2.5 all participated. None turned in a standout performance. The most common failure modes:
- Environment setup stalls immediately. Scientific computing has heavy dependencies—conda environments, CUDA versions, Python package binary compatibility. Any of these can send an Agent into an endless loop.
- Goal drift on long-horizon tasks. By step 30, the Agent has forgotten what it was doing at step 1.
- Verbose error messages overwhelm the model. Scientific tools produce extremely long error output, and Agents struggle to identify the relevant signal.
This doesn't mean these models are "bad." It means the complexity of a research workflow is an order of magnitude beyond writing code.
Writing code rewards pattern matching. Running a research experiment requires an Agent to simultaneously understand domain knowledge, tool stacks, data formats, and statistical significance—which is well beyond coding ability alone.
Why This Matters Now
In 2026, Agent evaluation is clearly trending away from "can it write code" and toward "can it solve real problems."
A few recent milestones:
- MLE-Bench: end-to-end reproduction of Kaggle competitions
- LAB-Bench: biological experiment design and interpretation
- ScienceBoard: multimodal scientific reasoning
Terminal-Bench-Science is one more piece along this trajectory. What sets it apart from the benchmarks above is "terminal-native" execution: the Agent must complete tasks inside a real command-line environment, not a controlled API wrapper with buttons. That distinction matters because real researchers work in terminals.
That said, Terminal-Bench-Science has obvious limitations:
- Task count sits at 80+, an order of magnitude below SWE-bench's 2,000+
- Domain coverage is narrow—only biology, materials science, and ML. Chemistry, physics, and earth sciences are untouched.
- The evaluation metric is single-dimensional, with no distinction between "full success" and "partial success"
So declaring "Terminal-Bench-Science is the SWE-bench for research Agents" is premature.
Some Thoughts
The real value of benchmarks like this isn't the leaderboard—it's surfacing where Agents genuinely fall short.
With Terminal-Bench-Science specifically, what stands out is not "another SOTA test" but three pain points that get deliberately amplified:
- Long-horizon reasoning is fragile. On workflows beyond 30 steps, Agents almost inevitably lose focus. The issue isn't single-step reasoning quality—it's that chaining multi-step reasoning together still hasn't been genuinely solved.
- Tool-call error tolerance is weak. One wrong character in a scientific toolchain can break everything. Agents lack strategies for fallback, rollback, and path-switching.
- Scientific common sense is missing. Many tasks fail not because the code is wrong, but because the Agent doesn't know "this approach is wrong for this domain." The code runs; the results are physically meaningless.
The research community needs a benchmark that distinguishes between "knows how to use tools" and "actually understands research" far more than it needs another 1% improvement on SWE-bench. Terminal-Bench-Science moves in that direction, but the task count and domain coverage are still too small for any conclusive judgment—we'll need another 6–12 months of data.
My prediction: within the next 12 months, at least three similar "domain-native benchmarks" will emerge (chemistry, earth sciences, astronomy). Their value won't be in ranking models—it'll be in pushing Agents to evolve from "programming tools" into "research tools."
Reference: Terminal-Bench-Science announcement