AI‑driven incident response risks distancing engineers from their infrastructure

AI‑assisted incident response: capabilities and concerns

When AI systems begin to resolve operational alerts automatically, the day‑to‑day engagement of site‑reliability engineers with their platforms is diminishing.

The author recounts that as an SRE at LinkedIn in 2012 he designed a prototype system that could heal itself and learn from prior incidents.

At that time AI capabilities were far from today’s standards, and the prototype remained experimental.

Now the described capabilities have become a reality in production environments.

Modern tools can inspect alerts, formulate hypotheses, query telemetry, correlate recent deployments, and even apply corrective actions without human intervention.

The author admires the convenience but voices a major concern: engineers are losing direct contact with their systems.

The more routine incidents are delegated to automation, the fewer opportunities human responders have to practice diagnosing and fixing problems.

When an ambiguous, high‑severity incident arrives that the AI cannot resolve, engineers may be ill‑prepared to take over.

"Automation leaves humans with the hardest incidents".

These AI‑assisted incident response tools are often labeled “AI SREs,” a term the author does not favor.

They feel especially magical when they handle a nighttime capacity issue without waking anyone.

However, routine incidents are the primary way responders safely develop intuition about system behavior and failure modes.

The automation paradox and lessons from aviation

Human‑factors researcher Lisanne Bainbridge described this paradox in her 1983 paper The Ironies of Automation.

She explained that automation reduces operators’ opportunities to practice routine work while still holding them responsible for abnormal situations.

She argued that operators therefore need to become more skilled and receive even more training than before automation.

The author predicts that average mean time to recovery (MTTR) for most incidents will decline thanks to AI‑assisted response.

He also predicts that resolution time for complex incidents will increase because responders will have lost touch with their systems.

Aviation provides a useful analogy for this dilemma.

In modern aircraft, automation handles much of the flight, yet pilots remain accountable for failures that automation cannot manage.

Rare events such as engine failures occur at fewer than one in‑flight shutdown per 100,000 engine flight hours.

A commercial pilot may complete an entire career without experiencing such a failure outside a simulator.

When TransAsia Airways Flight 235 suffered an engine propeller autofeather shortly after takeoff, the crew misidentified the problem and the aircraft crashed.

Pilots regularly rehearse rare emergencies in simulators, and US FAA regulations require captains to complete recurrent training or a proficiency check every six months.

Although most software incidents do not threaten lives, the same discipline of regular rehearsal is valuable for maintaining expertise.

Simulating incidents to keep engineers sharp

The technology that creates the automation challenge can also help mitigate it.

At Rootly, where the author works, a partnership with Uptime Labs brings realistic incident simulations to engineers.

In these simulations, engineers assume the incident commander role during a mock e‑commerce outage.

They use observability tools while coordinating with LLM‑powered stakeholders in Slack.

The scenario feels real, requiring investigation of incomplete information and clear communication under pressure.

Participants must keep the response organized while dealing with simulated CEO and customer‑support demands.

This practice builds the skills needed for high‑stakes, ambiguous incidents that automation cannot solve.

The author suggests that such simulators could become a standard component of SRE training programs.

By regularly exercising rare failure modes, engineers can preserve intuition even as AI handles routine work.

Why This Matters: As AI automates routine outages, deliberate incident‑simulation training will be essential to keep engineers capable of handling the rare, high‑impact failures.