AI Engineering5.0 · 50 ratings

LLM Ops Runbook

**Role:** SRE specialized in LLM-powered services. **Context:** Team is on-call for LLM features but has no runbook. Incidents take hours t…

Role-BasedChain-of-Thought

Prompt

**Role:** SRE specialized in LLM-powered services.

**Context:** Team is on-call for LLM features but has no runbook. Incidents take hours to diagnose because everyone's debugging from scratch.

**Task:** Build the runbook:
1. **Alert: high latency.** Diagnosis steps, common causes (provider outage / queue backlog / model upgrade), mitigations.
2. **Alert: high cost.** Diagnosis (runaway loop / new traffic / model drift), mitigations.
3. **Alert: refusal rate spike.** Causes (prompt change / model change / hostile traffic), mitigations.
4. **Alert: hallucination rate spike.** Causes, mitigations.
5. **Alert: tool failure rate spike.** Causes, mitigations.
6. **Alert: provider 5xx surge.** Failover protocol.
7. **Alert: customer-visible regression.** Rollback protocol, comms template.
8. **Postmortem template** for LLM-specific incidents.

**Constraints:**
- Every entry has named owners + escalation chain.
- Mitigations have time bounds (try X for 5 min, then escalate).
- Rollback procedures are tested quarterly.

**Output format:** Runbook markdown + on-call cheatsheet + postmortem template.

How to use this prompt

  1. 1

    Copy the prompt above and paste it into ChatGPT, Claude, or Gemini — or open it in the visual Studio to edit each part on a canvas and run it with your own key.

  2. 2

    Replace any bracketed placeholders with your specifics. The more concrete your context and constraints, the sharper the result — see the 5-part prompt structure.

  3. 3

    Run it, then refine. Ask the model to critique and improve its own answer with self-critique prompting.

Techniques in this prompt

Role-Based

Assigns the model an expert persona so it adopts the right vocabulary, depth, and standards for the task.

Learn this technique
Chain-of-Thought

Asks the model to reason step by step before answering — ideal for multi-step, logical, or analytical tasks.

Learn this technique

Recommended models

claudegpt-4o

Build on this prompt

Open it in the visual Studio to wire it into a full workflow with your own API key — or learn the craft behind prompts like this.

Open in Studio How to prompt AI correctly

More in AI Engineering

RAG vs Fine-tune Decision Memo

**Role:** You are a senior AI engineer who has shipped both RAG-based and fine-tuned LLM products at production scale. You believe most team…

Read prompt

Evals Harness Design for [Domain]

**Role:** AI engineer who has built evals suites that have caught 30+ production regressions before they shipped. You believe vibes-based "t…

Read prompt

System Prompt Audit

**Role:** Senior prompt engineer who has audited 100+ production system prompts. You read prompts the way an editor reads prose — for the me…

Read prompt

Agent Loop Halt-Condition Design

**Role:** Applied AI engineer who has shipped agents that completed millions of tool-calling iterations in production. You believe most agen…

Read prompt