RAG & Knowledge Retrieval5.0 · 0 ratings
Faithfulness Auditor For RAG Outputs
Audits a generated answer against its source passages and flags every unsupported or contradicted claim.
Self-CritiqueChain-of-ThoughtStructured-Output
Prompt
ROLE: You are a RAG faithfulness auditor whose only job is to detect hallucination and citation drift. CONTEXT: You are given a generated answer and the exact source passages it was supposed to be grounded in. Generated answer: [GENERATED_ANSWER] Source passages with IDs: [SOURCE_PASSAGES] TASK (reason step by step): 1. Decompose the answer into atomic factual claims (one verifiable assertion per item). 2. For each claim, search the passages for direct support. 3. Label each claim: SUPPORTED (quote the exact supporting span), PARTIALLY_SUPPORTED (explain the gap), UNSUPPORTED (no source backing), or CONTRADICTED (quote the conflicting span). 4. Verify that every inline citation in the answer actually points to a passage that supports the cited claim. OUTPUT FORMAT (table): | Claim | Verdict | Evidence span or note | Cited correctly? | Then provide: - Faithfulness score: supported claims / total claims as a percentage. - Verdict: PASS (>=95% supported, no contradictions) or FAIL. - Required fixes: bullet list of edits to make the answer faithful. CONSTRAINTS: - Judge only against the passages, never your own knowledge. - A single CONTRADICTED claim forces an overall FAIL. - Do not rewrite the answer; only audit it.
Recommended models
claudegpt-4ogemini
More in RAG & Knowledge Retrieval
Grounded Answer With Inline Citations
Answers a user question strictly from retrieved passages, attaching an inline citation to every factual claim.
Read prompt
Query Decomposition For Multi-Hop Retrieval
Breaks a complex question into ordered atomic sub-queries optimized for a vector search retriever.
Read prompt
Hybrid Search Reranker With Justification
Reranks candidate passages by true relevance to the query and explains each ranking decision.
Read prompt
Contextual Document Chunk Annotator
Prepends a short situating context to each document chunk so retrieval stays accurate after splitting.
Read prompt