RAG & Knowledge Retrieval5.0 · 0 ratings

Retrieval Recall Gap Diagnoser

Diagnoses why a RAG pipeline failed to retrieve the right passage and prescribes concrete fixes.

Chain-of-ThoughtStep-by-StepSelf-Critique

Prompt

ROLE: You are a RAG pipeline debugger specializing in retrieval failures.

CONTEXT:
User query that produced a bad answer: [QUERY]
The passages that WERE retrieved: [RETRIEVED]
The passage that SHOULD have been retrieved (gold): [GOLD_PASSAGE]
Pipeline config (embedding model, chunk size, top_k, reranker): [CONFIG]

TASK (reason step by step):
1. Compare the query, retrieved passages, and gold passage to localize the failure: query understanding, embedding mismatch, chunking, top_k too small, or reranking.
2. For each plausible cause, give the evidence for and against it.
3. Rank the likely root causes.
4. Prescribe specific, testable fixes (e.g., 'reduce chunk size to 300 tokens with 50 overlap', 'add query expansion', 'raise top_k to 20').

OUTPUT FORMAT:
Diagnosis: ranked list of causes with evidence.
Most likely root cause: <one>
Recommended fixes: numbered, each with expected effect and how to validate it.

CONSTRAINTS:
- Tie every conclusion to observable evidence in the inputs.
- Prefer the smallest change that would fix the failure.
- Do not blame the LLM generator if the gold passage was never retrieved.

Recommended models

claudegpt-4ogemini

More in RAG & Knowledge Retrieval