Case Studies

GPT-5 Codex Runs Real Quantum Experiments — But What Did OpenAI Actually Show?

OpenAI posted a case study on using Codex to assist live quantum computing research. No benchmarks, no controls — just a showcase that reveals how far agentic AI still is from real lab work.

Font size

OpenAI published a case study yesterday: Codex in quantum computing experiments. Not a benchmark, not a paper — a "look, we let AI help researchers do their jobs" showcase.

After reading it, the main thing it exposes isn't how capable Codex is. It's how far "AI in the lab" still is from real deployment.

The Case Itself: What Codex Actually Did

OpenAI partnered with a research team running superconducting qubit experiments on IBM Quantum hardware and plugged Codex into their workflow. According to the post, the AI handled three main tasks:

  1. Writing experiment control code — generating Qiskit code from natural language descriptions and scheduling pulse sequences.
  2. Tuning and sweeping parameters — suggesting the next parameter set based on the previous run's results.
  3. Interpreting experimental data — reading raw data and flagging anomalies.

That's the workflow shown in the post. Researchers didn't disappear — they shifted from writing code to reviewing code and asking questions.

Why Quantum Computing?

OpenAI didn't pick this case randomly. Quantum experiments have several properties that match an AI agent's capability profile:

  • High coding barrier: Qiskit, Cirq, pulse scheduling — getting started takes weeks if you've never done it.
  • Explosive parameter space: A superconducting qubit experiment with gate amplitudes, phases, and durations can yield thousands of combinations.
  • Painful debugging: Each hardware run requires queuing time.
  • Quantifiable results: Metrics like fidelity, T1, and T2 are clean and give AI something solid to interpret.

In other words, it's a scenario where AI looks promising and humans genuinely struggle. OpenAI's choice to showcase it sends a clear message: Codex isn't limited to writing CRUD apps — it can serve as an assistant in high-barrier technical domains.

The subtext is obvious: they're pitching to the research market. The GPT-5 line is trying to move from ChatGPT's consumer conversations into B2B, and research labs are a budget-insensitive segment with strong willingness to pay.

What 11 Upvotes and 0 Comments on HN Tell Us

The post made it to the Hacker News front page. I checked the numbers: 11 points, 0 comments.

That number is telling. Two years ago, an OpenAI case study like this would have drawn dozens of discussion threads. Zero comments now means one thing: readers are getting fatigued by the "AI does science" narrative.

The fatigue isn't because AI doesn't work. It's because the story is overplayed. From AlphaFold to GPT-4 reading radiology reports, "AI in the lab" gets trotted out every few months. Everyone already gets it. The question has shifted from "can it?" to "how fast, how cheap, how scalable?"

OpenAI's post doesn't answer those three questions. So nobody commented.

What's Missing: Why This Isn't Persuasive

Torn apart by strict standards, the post is missing at least three things:

1. No benchmark.
Was the experiment fidelity 99.2% or 98.7% with Codex-written code? Did the final experimental results differ from human-written code? Nothing.

2. No control group.
How long does the traditional workflow (human writes code → runs experiment → interprets results) take? How long with Codex? What's the speedup? No data.

3. No scale.
Is "AI helping researchers" a one-off case or a weekly routine? How many experiments per month? Public materials don't say.

This style of writing is called a showcase in marketing, a demo in engineering. The difference: a showcase makes you say "wow," a demo lets you reproduce the result. OpenAI delivered the former.

Not trying to dunk on them — a blog post can't carry a full study. But readers should keep in mind: what they're looking at isn't evidence, it's an announcement.

Editor's Take: The Real Bottleneck for AI in Labs Isn't the AI

I'm an engineer who has shipped a few LLM integrations into real production systems. After reading the post, my take is: model capability crossed the threshold a while ago. The real bottleneck is outside the AI.

Three specific blockers:

  • Lab equipment interfaces: Quantum computers, superconducting cavities, electron microscopes — vendors are closed-source, protocols are proprietary. Codex can write code, but that code can't run — IBM Quantum's backend API isn't openly accessible, and you're waiting in queue anyway.
  • Data formats: Every instrument spits out raw data in a different format. Codex can read markdown, but it can't parse a binary pulse dump.
  • Lab procedures: Safety approvals, equipment scheduling, reagent and consumables management — AI has zero hand in any of this.

None of these three things get mentioned in OpenAI's post.

Zooming out, here's how this case differs from previous efforts: every "AI does science" case from the past two years has essentially been "code generation + literature search" — the things LLMs are best at. The quantum experiment case goes one step further: it lets AI touch real hardware. But it's still just "generating code that submits jobs," not actually controlling the instrument.

My prediction: over the next 12 months, the research agents that actually ship won't be combinations of general-purpose models and general-purpose APIs like OpenAI's. They'll be small models trained by vertical labs, tightly integrated with instrument SDKs. The reason is simple: hardware isn't general-purpose.

If I'm wrong, feel free to dig this up in six months.


Source: OpenAI Codex Quantum Computing Experiments

Source: https://openai.com/index/codex-quantum-computing-experiments/