AI Engineering5.0 · 50 ratings

LLM Failure-Mode Taxonomy

**Role:** AI safety researcher building the team's shared vocabulary for LLM bugs. **Context:** Team is shipping LLM features fast and prod…

Role-BasedChain-of-Thought

Prompt

**Role:** AI safety researcher building the team's shared vocabulary for LLM bugs.

**Context:** Team is shipping LLM features fast and producing inconsistent bug reports. Engineering and QA don't have a shared vocabulary for "what went wrong."

**Task:** Build the taxonomy:
1. **Hallucination** types: factual confabulation, citation invention, format invention.
2. **Refusal** types: over-refusal, under-refusal, miscalibrated refusal.
3. **Drift** types: persona drift, format drift, scope drift.
4. **Reasoning** failures: shallow CoT, math errors, contradiction tolerance.
5. **Tool-use** failures: wrong tool, wrong args, ignored output.
6. **Format** failures: invalid JSON, broken markdown, encoding mismatch.
7. **Latency / cost** failures: token waste, slow tool calls, over-reasoning.
8. **Safety** failures: PII leakage, jailbreak success, copyright leak.

For each: definition, example, observable signal in logs, who's responsible for fixing.

**Constraints:**
- Every category has a CONCRETE EXAMPLE from real production.
- Each failure has a single "owner" team.
- Avoid academic terms when ops terms exist.

**Output format:** Taxonomy doc + bug-template (Jira / Linear / GitHub) using these labels.

How to use this prompt

  1. 1

    Copy the prompt above and paste it into ChatGPT, Claude, or Gemini — or open it in the visual Studio to edit each part on a canvas and run it with your own key.

  2. 2

    Replace any bracketed placeholders with your specifics. The more concrete your context and constraints, the sharper the result — see the 5-part prompt structure.

  3. 3

    Run it, then refine. Ask the model to critique and improve its own answer with self-critique prompting.

Techniques in this prompt

Role-Based

Assigns the model an expert persona so it adopts the right vocabulary, depth, and standards for the task.

Learn this technique
Chain-of-Thought

Asks the model to reason step by step before answering — ideal for multi-step, logical, or analytical tasks.

Learn this technique

Recommended models

claudegpt-4o

Build on this prompt

Open it in the visual Studio to wire it into a full workflow with your own API key — or learn the craft behind prompts like this.

More in AI Engineering