AI Engineering5.0 · 50 ratings

Output Safety Classifier

**Role:** Trust & Safety ML engineer. **Context:** Need to classify LLM outputs as safe / unsafe before returning to users. Can't rely sole…

Role-BasedChain-of-Thought

Prompt

**Role:** Trust & Safety ML engineer.

**Context:** Need to classify LLM outputs as safe / unsafe before returning to users. Can't rely solely on the model's own refusal.

**Task:** Design the classifier:
1. Output categories (forbidden / sensitive / safe).
2. Classifier choice (rules / ML model / LLM-as-judge).
3. Training data (positive + negative examples).
4. False-positive / false-negative tradeoff.
5. Latency budget.
6. Calibration with human review.
7. Action on flagged outputs (block, modify, log, escalate).
8. Evaluation rubric.

**Constraints:**
- p95 classifier latency ≤ 50ms.
- False-negative on critical categories ≤ 0.5%.
- All flags reviewable in an audit log.

**Output format:** Architecture + training-data spec + evaluation plan.

How to use this prompt

  1. 1

    Copy the prompt above and paste it into ChatGPT, Claude, or Gemini — or open it in the visual Studio to edit each part on a canvas and run it with your own key.

  2. 2

    Replace any bracketed placeholders with your specifics. The more concrete your context and constraints, the sharper the result — see the 5-part prompt structure.

  3. 3

    Run it, then refine. Ask the model to critique and improve its own answer with self-critique prompting.

Techniques in this prompt

Role-Based

Assigns the model an expert persona so it adopts the right vocabulary, depth, and standards for the task.

Learn this technique
Chain-of-Thought

Asks the model to reason step by step before answering — ideal for multi-step, logical, or analytical tasks.

Learn this technique

Recommended models

claudegpt-4o

Build on this prompt

Open it in the visual Studio to wire it into a full workflow with your own API key — or learn the craft behind prompts like this.

More in AI Engineering