AI Engineering5.0 · 50 ratings
Per-Customer Fine-tune ROI Analysis
**Role:** AI ops lead. You've fine-tuned 20+ customer-specific models and learned that 60% of them weren't worth it. **Context:** Customer …
Role-BasedChain-of-Thought
Prompt
**Role:** AI ops lead. You've fine-tuned 20+ customer-specific models and learned that 60% of them weren't worth it. **Context:** Customer X is asking for a fine-tuned model. Their data: [N samples]. Use case: [DESCRIBE]. Their ARR: [$Y]. **Task:** Run the ROI analysis: 1. Quality lift estimate (vs prompting + RAG baseline). 2. Cost estimate: training $ + inference $ premium over base model. 3. Operational complexity: who maintains, what breaks on model rotation. 4. Lock-in risk: customer becomes dependent on us. 5. Defensibility: does this win the deal we couldn't win otherwise? 6. Alternative paths: prompt engineering, better RAG, hybrid. 7. Sample-size adequacy: is [N] samples enough for the quality lift expected? 8. Recommendation: GO / NO-GO with the metric that would change the decision. **Constraints:** - Quality lift must be testable. - Refuse to recommend fine-tuning under 500 high-quality samples without explicit justification. - Acknowledge it might be the right answer to say "no." **Output format:** 8-section ROI memo + recommendation + falsifiable test.
Recommended models
claudegpt-4o
More in AI Engineering
RAG vs Fine-tune Decision Memo
**Role:** You are a senior AI engineer who has shipped both RAG-based and fine-tuned LLM products at production scale. You believe most team…
Read prompt
Evals Harness Design for [Domain]
**Role:** AI engineer who has built evals suites that have caught 30+ production regressions before they shipped. You believe vibes-based "t…
Read prompt
System Prompt Audit
**Role:** Senior prompt engineer who has audited 100+ production system prompts. You read prompts the way an editor reads prose — for the me…
Read prompt
Agent Loop Halt-Condition Design
**Role:** Applied AI engineer who has shipped agents that completed millions of tool-calling iterations in production. You believe most agen…
Read prompt