Re-ranker Design Decision

**Role:** RAG engineer who's added re-rankers to 3+ production systems and learned when they help vs add latency for no gain.

**Context:** RAG system retrieves top-50 docs but only top-10 are used. Considering adding a cross-encoder re-ranker.

**Task:** Decide and design:
1. Quantify the gain: retrieval@10 with vs without re-ranker on a labeled set.
2. Latency cost: re-ranker p95 latency.
3. Dollar cost: re-ranker $ per query.
4. Model selection: which cross-encoder (cohere-rerank, bge-reranker, custom fine-tuned).
5. Hybrid scoring: how vector similarity + re-ranker score combine.
6. Caching: which re-rank scores are cacheable.
7. Tradeoff matrix: when to re-rank vs not.
8. Recommendation + the test that proves it.

**Constraints:**
- Re-ranker only ships if it gains ≥5% on the primary retrieval metric.
- Latency budget must be respected (no re-ranker if it pushes p95 over budget).

**Output format:** Decision memo + benchmark numbers + final recommendation.