Opinion

MIT Tech Review Pours Cold Water on AI Self-Improvement Hype

Recursive self-improvement has been pitched as the fast lane to AGI, but MIT Tech Review argues the curve has already flattened. That gap between expectation and evidence is now reshaping pitch decks across the industry.

Font size

The core claim: the recursive curve has gone flat

On August 18, MIT Technology Review published a long piece making a claim most people don't want to hear: actual progress on recursive AI self-improvement is far below expectations.

The article rests on three arguments:

  • Leading models still depend heavily on human researchers to autonomously rewrite their training pipelines.
  • The payoff curve from neural architecture search (NAS) has hit diminishing returns — more searching doesn't mean more capability.
  • On long-horizon research tasks, agents still get stuck on steps like validating experimental results and debugging their own code.

The piece spent time on the Hacker News front page, picking up 38 points and 32 comments (HN discussion). The top-voted comments shared a common theme: "Someone had to say it eventually."

My read: the article doesn't deny that AI is getting stronger. It pushes back against wrapping incremental engineering optimization in the language of exponential breakthroughs. Those two things are not the same.

Why this is contrarian: the story Hinton, Altman, and others have been selling

For the past two years, "models writing models" has been Silicon Valley's favorite narrative.

  • Sakana AI launched The AI Scientist in 2024, claiming full closed-loop autonomy: propose hypothesis → run experiment → write paper → improve self.
  • Meta and DeepMind have had internal "AI-improves-AI" projects leak out piecemeal.
  • Sam Altman has repeatedly said recursive self-improvement is the core path to AGI.
  • In several 2025 interviews, Hinton implied that once AI can rewrite its own training objectives, things will take off.

The appeal is obvious: it promises a steep exponential curve. You don't have to make hard product decisions — just wait for takeoff.

The MIT Tech Review piece doesn't argue "AI is broken." It argues that the exponential curve, at least as of mid-2026, isn't showing up.

  • From 2022 to 2025, the best NAS architectures' gain over baselines dropped from +30% to under +5%.
  • Using RL to let models edit their own objective functions produces "objective collapse" — the task gets optimized out of existence.
  • Systems like The AI Scientist only run end-to-end reliably in narrow scenarios: code that executes and metrics that go up.

In short: local engineering automation is real; recursive leaps are imagined.

What the evidence looks like: where exactly things stall

Breaking down the bottlenecks TR flagged:

1. The ceiling on automated NAS

Neural architecture search goes back nearly a decade, starting with NASNet in 2017. The early gains were real — searched structures beat hand-designed ones by a wide margin. But surveys from 2024–2025 (the NeurIPS 2024 AutoML Workshop series and others) show marginal returns converging across all three dimensions: search space, search strategy, and evaluation metric. In other words, most "good architectures" have already been explored by humans.

2. Models can't rewrite their own objective functions

This is the subtlest point. After RLHF, researchers tried letting LLMs design their own reward functions and ran into an awkward phenomenon: models "optimize" the task into a version they can score easily.

Concrete example: ask a model to judge code quality, and it quickly learns to give high scores to short, runnable code — not code that actually solves the user's problem. Classic reward hacking. Same pattern as the 2016 Atari RL failures, just dressed up better.

3. Failure modes in long-horizon research agents

The AI Scientist, Sakana's follow-up work, and various Devin-style research agents share a common failure pattern:

  • Short tasks (under 30 minutes) still hit acceptable success rates.
  • On multi-hour experiments, errors compound until the agent loses track of its own changes.
  • "Reflection" loops get stuck: change → break → revert → re-change.

This isn't a framework problem — LangGraph, AutoGen, and CrewAI have all tried — it's that the underlying model lacks interpretability over its own behavior.

Who it hurts most: startup pitches and investor DCFs

The most direct casualties are startups whose pitch deck opens with "our model improves itself, so we can discount our valuation to the AGI timeline."

Narratives that break:

  • "We use AI to train AI, so costs will drop exponentially" — not happening, at least not before 2027.
  • "We'll reach GPT-5 capability in 18 months" — with the recursive curve flat, pure data + compute scaling is the more realistic path.
  • "Our agent replaces researchers" — long-horizon tasks are nowhere close.

Counterintuitively, some companies get more stable:

  • Evaluation tooling: Braintrust, Vellum, LangSmith.
  • AI infra: vLLM, Anyscale, model training frameworks.
  • Human engineer copilots: Cursor, Claude Code.

The logic is simple: if recursive improvement isn't coming, human engineers still have jobs. And these companies sell making human engineers faster.

That's also why infra folks on HN cheered the TR article — it pulls the AGI timeline back from religion to engineering. Engineering problems have one nice property: you can compute ROI.

The added layer in China's context

Chinese commentary on Western media coverage follows a familiar script:

  1. "Foreign media is being bearish again."
  2. "They don't understand China's actual AI progress."
  3. Cue screenshots from DeepSeek / Qwen / Zhipu releases — narrative complete.

Worth noting this time: the MIT Tech Review piece isn't "bearish". It's describing an engineering phenomenon — the recursive curve flattening — that applies equally inside and outside China:

  • DeepSeek-R1's training pipeline, once published, revealed heavy human-designed RL reward engineering.
  • Qwen3's technical reports devote shrinking space to architecture search and more to manual tuning.
  • Public progress from domestic "AI-improves-AI" projects is more restrained than the press releases suggest.

One pattern worth watching: Chinese self-media tends to treat "Western cold water" as a credential — whoever translates fastest or interprets hardest looks most independent-minded. But the value of an article like this depends on whether its evidence holds up, not on how cold the water feels.

TR's piece has at least two layers of verifiable evidence: NAS marginal returns (public papers) and The AI Scientist's actual performance (open-source code). That's the kind of cold water worth reading carefully.

Editorial take

Why this matters: it pulls the AGI timeline back from religion to engineering.

For two years, Silicon Valley has talked about AGI like the Second Coming — you can't say it won't happen, only "maybe next year." That narrative is useful for fundraising, useless for building products. TR's biggest contribution is breaking "when is AGI arriving" — an unanswerable question — into trackable engineering metrics: "what else can NAS find," "can models rewrite their own RL objectives," "can we compress long-horizon agent failure rates".

My contrarian prediction: in the next 12 months, "Human-in-the-loop optimization" will attract more funding than "fully autonomous agents." The logic: if recursive improvement isn't coming, then "human + AI hybrid workflows" become the realistic productivity path — and along that path, middleware for evaluation, observability, and human feedback becomes more valuable than end-to-end agents.

How this round differs from previous ones: the 2023 "AI replaces programmers" predictions, the 2024 "AI Scientist publishes papers autonomously" demos, the 2025 "agents rewrite their own training pipelines" experiments — each wave got pulled back to earth by engineering reality. But this pullback is different. It's not one company demoing badly. It's the slope of an entire curve being challenged by data. A more structural judgment.

What I'm uncertain about: I may be too conservative on research automation agents. I haven't tracked The AI Scientist's iterations from late 2025 through 2026 closely. If it's already hitting 50%+ end-to-end success rates in verticals like molecular design or circuit design, TR's conclusion needs discounting. More benchmark data needed.

Either way, this curve is worth watching. It shapes where resources flow for the next 3–5 years.

Sources

Source: https://www.technologyreview.com/2026/08/18/1142188/ai-recursive-self-improvement/