GuideAI in production
How to evaluate incident-response agents
Build repeatable cases and check recovery independently, with a runnable Python grader.
Articles
Guides, news and explanations about AI and reliable systems.
GuideAI in production
Supervising coding agents with Herdr
How Herdr keeps coding sessions available while you detach, and what changes when its server restarts.
GuideIncident response
What did the postmortem fix prove?
Separate a completed task from evidence that the underlying risk was reduced.
GuideIncident response
Why a 5 Whys investigation can branch
Evidence can split a 5 Whys investigation into several contributing causes. A concurrency example shows how to turn those explanations into useful tests.
GuideAI in production
Comparing incident sources with NotebookLM
Conflicting restart instructions show where a source-backed notebook helps: finding the relevant passages, dates and exceptions for the runbook owner.
GuideAI in production
Evaluating Claude Opus 4.6 for incident work
A rollback exception tests whether Opus 4.6 preserves decisive evidence across a long session, and whether the resulting answer arrives in time to help.
GuideAI in production
Recovering an interrupted agent skill
A timed-out scaling request needs a durable record and destination-state checks. Reusable operational skills need to recover unfinished work as well as start it.
The on-call work that page counts leave out
Disturbed nights, specialist help and cleanup consume capacity beyond the primary shift. A workload worksheet connects that demand to staffing and delivery plans.
GuideSLOs & reliability
Release gates that hold up under incident pressure
An error-budget hold needs a route for an urgent repair. Fresh measurements, artifact-specific exceptions and recovery criteria determine how that route works.
GuideIncident response
Keeping incident state consistent across PagerDuty, Jira and Slack
A reference design for duplicate deliveries and partial API success, with durable incident identifiers and summaries drawn from the resulting record.
GuideObservability
Grafana dashboards for incident response
Regional failures can disappear in fleet averages. Grafana queries, population labels and linked traces help responders find the affected requests.
GuideAI in production
Prompting an assistant to investigate an incident
A reusable prompt connects observations, competing explanations and the next read-only check, so an incomplete record can still support useful investigation.