Opinion

AI Is Handling On-Call. Now Engineers Are Losing Their Feel for the System

A Hacker News thread hit 84 points after Sylvain Kalache argued that AI-driven incident response is eroding SRE intuition. The comments split into camps, and the aviation industry's old lessons are starting to replay in site reliability.

Font size

Last week a blog post climbed to 84 points and 51 comments on Hacker News. The author, Sylvain Kalache (ex-SRE, now a Red Hat consultant), made a blunt argument:

AI is increasingly handling production incidents, but engineers are losing the intuition and muscle memory for how their systems actually behave.

Half the comment section said "finally someone said it." The other half shared war stories: "the last time I manually debugged this was three years ago." This wasn't one person venting — it was a group collectively admitting that automation is reshaping the SRE profession.

Sources: HN thread, Sylvain's original post.

What the HN thread was actually arguing about

The comments sorted into a few camps:

  • The "we already live this" camp: Someone shared internal incident retrospectives showing the same database connection pool exhaustion had hit four times in five years. The last two were handled entirely by AI — engineers just watched the reasoning scroll past in Slack.
  • The "this is Luddite thinking" camp: Cars replaced horses. CI replaced manual deploys. People cried about skill atrophy both times. Society survived.
  • The "it depends" middle: Batch jobs can go fully autonomous. Interactive debugging needs a human in the loop.

Who's right matters less to me than the vocabulary. "Muscle memory," "intuition," "feel for the system" — these phrases barely showed up in SRE discussions five years ago. The fact that they show up now means something is genuinely draining out of the profession.

The "automation paradox" isn't new — but this round is different

Automation complacency originated in aviation psychology: pilots who lean on autopilot long enough lose measurable manual flying proficiency. After a string of 1990s crashes (Air France 447 among them), regulators mandated a minimum number of manual flight hours every 90 days to fight the effect.

Medicine has a parallel conversation. Clinical decision support systems (CDSS) have been in use for over two decades, and there's evidence that diagnostic intuition for rare conditions is getting duller. A 2019 study on radiologists found that doctors who had spent years using AI-assisted imaging scored roughly 10% lower on unaided reads.

But SRE is fundamentally different from both fields.

Pilots and doctors operate systems built by someone else (airframes, imaging machines). Knowledge loss is somewhat inevitable. SREs operate systems their own teams built. Theoretically, you wrote the code, designed the architecture, and maintain the runbooks. The knowledge lives inside the team. There's no "unknowability of an external system."

AI breaks that chain.

When an AI agent handles an incident, the logs it reads, the queries it runs, and the runbook steps it executes mostly pass through a model instead of a human brain. Engineers shift from diagnostician to auditor — glance at the AI's report, sign off, close the ticket. In the short term, MTTR drops. In the long term, blind spots grow in how engineers understand their own systems.

That's what's different this time. Automation itself isn't the problem. The way it's being applied bypasses the human learning loop entirely.

How muscle memory disappears: the hidden cost of on-call

Concretely, how does this fade? Three common scenarios.

Scenario 1: Automatic alert suppression.

Old workflow: a disk alert fires, the on-call engineer SSHes in, runs df -h, then du -sh /*, tracks down which log directory exploded, and either cleans it or expands capacity. Fifteen to thirty minutes. Do it once and you remember that directory's temperament for years.

New workflow: an AI agent runs the same command chain, posts "Cleaned /var/log/app/, freed 12 GB" to Slack. Engineer reads, closes the incident. The ticket is resolved on paper. The engineer learned nothing.

Scenario 2: Automatic root-cause analysis.

Old workflow: a Prometheus + Alertmanager storm means cross-referencing metrics, chasing traces, and guessing which service and which commit introduced the regression. Painful — and valuable. It forces you to internalize service dependencies.

New workflow: the AI agent queries Datadog (or an in-house AIOps platform) and outputs "Root cause: svc-order CPU anomaly after commit abc123." The engineer clicks confirm or rollback.

Scenario 3: Runbooks become APIs.

This is the most dangerous step. A runbook used to be a checklist for humans. Now it's an API surface for AI. But the value of a runbook was never just the steps — it was writing the runbook. Writing one means replaying the incident, listing preconditions, thinking through edge cases. That thinking disappears when a machine just executes it.

China's big tech companies deployed AIOps early. ByteDance's smart alerting, Alibaba Cloud ARMS, and Meituan's CAT (monitoring and alerting platform) all went large-scale around 2022–2023. Several SRE friends in China have told me the same thing privately: on-call is easier, but during promotion interviews, when asked "where's the deepest bottleneck in your system," they suddenly can't answer.

When the AI goes down, who takes over?

The sharpest point in Sylvain's post is here:

When the AI itself fails or becomes unavailable, engineers may no longer be able to step back in.

This isn't science fiction. During CrowdStrike's global outage in 2024, many organizations discovered that their recovery procedures depended so heavily on vendor tooling that the manual fallback path was rusty at best.

Translate that to an SRE context:

  • Your AIOps platform depends on an LLM API that gets rate-limited or goes down (OpenAI outage, or cross-border model access issues in China).
  • The AI's "root cause" is wrong, but nobody catches it because everyone assumes the AI is right.
  • The runbook steps the AI executes make the incident worse — and the on-call engineer no longer knows how to intervene manually.

Any one of these is a production fire.

Commenters offered some solutions. Two stood out:

  • An "AI-free day" drill. Once a week or month, fully disable AI assistance and run all incidents the old-fashioned way. MTTR will spike temporarily. The team's muscle memory gets preserved.
  • Reverse postmortem auditing. For every AI-handled incident, an engineer must manually replay the investigation afterward — not by reading the AI's report, but by running commands, reading logs, and arriving at their own conclusion. When their conclusion differs from the AI's, dig into why.

Both ideas are essentially the same trick aviation uses with its manual flight-hour requirements.

Have China's big tech AIOps deployments reached this point?

ByteDance, Alibaba, and Meituan's smart alerting systems have been live for two to three years. From public engineering blogs, they all cover roughly the same ground:

  • Time-series anomaly detection (statistical models or lightweight ML)
  • Alert correlation and noise reduction (compressing thousands of daily alerts down to dozens)
  • Root-cause localization (graph algorithms + service dependency topology)
  • Partial automatic remediation (K8s pod restarts, traffic shifting)

But fully autonomous incident response is still rare inside Chinese big tech. Most systems stop at "AI suggests, human confirms." That gives Chinese engineers a small buffer — muscle memory may be eroding a beat slower than in Silicon Valley.

That buffer won't last. Once LLM agent frameworks (LangGraph, AutoGen, CrewAI) mature enough to reliably chain multi-step tool calls, the "human confirms" step will get cut.

Editor's note

Why this matters: it's not another abstract "AI replaces humans" narrative. It's a quiet, specific kind of bleeding happening in one profession (SRE). The transition from diagnostician to auditor is silent. Nobody screams until the day the AI itself goes down.

A contrarian take: most people frame this as "skill degradation," but I'd argue it's "skill definition migrating." In ten years, "understanding systems" might mean understanding AI toolchains, not Linux kernels — the same way nobody today thinks skipping tape drives means you've lost your engineering soul. It's not decay. It's generational mismatch during a migration.

What's different from past shifts: compilers replaced mechanical labor (lexical analysis). CI replaced repetitive labor (running tests). AIOps is the first time judgment itself gets replaced — even "what kind of incident is this" is now decided for you. This is a qualitative leap, not a quantitative one.

What's unresolved: an "AI-free day" almost certainly won't land in cost-pressured small and mid-sized teams. Only well-staffed SRE organizations at big companies can afford it. So the real question isn't whether to preserve muscle memory — it's whether organizations will pay for a capability they might never use. Aviation reached that consensus after crashes. SRE may not be willing to pay the same tuition.

Source: https://www.sylvainkalache.com/blog/ai-handles-incidents-engineers-lose-touch-with-their-systems