Run as a team
Workflow catalog
Reference playbooks for the closed loop. Each entry is a real, reproducible incident pattern that ships in the repo or is described in docs/workflow-catalog.md.
How to read a workflow
Each workflow lists the phases the loop will run through. A workflow that omits critiqued means severity is below critical and the critic is skipped. A workflow that omits postmortem means the change was below the postmortem threshold.
Catalog
| Workflow | Family | Severity | Summary |
|---|---|---|---|
pg-connection-pool-exhaustion detectedcorrelatedinvestigatedcritiquedapprovedrecoveredverifiedpostmortem | capacity | critical | Detect connection_pool_used_ratio > 0.9 across replicas, correlate to recent deploys, propose narrowing the pool floor. |
pg-disk-io-saturation detectedcorrelatedinvestigatedapprovedrecoveredverified | disk | critical | Detect sustained disk_io_utilization > 95%, correlate to checkpoint storms, propose IOPS throttling or WAL tuning. |
pg-lock-wait-long-transaction detectedcorrelatedinvestigatedcritiquedapprovedrecoveredverifiedpostmortem | lock | error | Detect lock waits > 30s, identify the head blocker, propose cancellation with explicit blast radius. |
pg-replica-replay-lag detectedcorrelatedinvestigatedapprovedrecoveredverified | replication | warn | Detect replica replay lag > 60s, propose failover or read-only routing depending on the recovery budget. |
redis-eviction-storm detectedcorrelatedinvestigatedapprovedverified | cache | warn | Detect eviction rate spike, correlate to recent key TTL changes, propose policy tuning. |
Extending the catalog
Add a new workflow by contributing a YAML file under workflows/ and a matching scenario under deploy/incident-events/. PRs are welcome — see CONTRIBUTING.md.