AI Engineering5.0 · 50 ratings
Vendor LLM Evaluation
**Role:** AI vendor selection lead. **Context:** Team evaluating LLM providers (OpenAI / Anthropic / Google / Mistral / self-hosted). Need …
Role-BasedChain-of-Thought
Prompt
**Role:** AI vendor selection lead. **Context:** Team evaluating LLM providers (OpenAI / Anthropic / Google / Mistral / self-hosted). Need rigorous comparison. **Task:** Eval: 1. Eval criteria: capability, cost, latency, compliance, vendor lock-in, SLA, support, roadmap. 2. Per-vendor scoring across each dimension. 3. Capability benchmark on YOUR use case (not generic MMLU). 4. Cost projection for YOUR scale. 5. Compliance: SOC 2, HIPAA, EU residency, indemnification. 6. Switch cost: how to migrate off this vendor later. 7. Recommendation + backup. 8. Re-evaluation schedule. **Constraints:** - Use-case-specific benchmark, not industry-standard. - Cost is your real projected cost, not list price. **Output format:** Vendor comparison table + recommendation + migration plan.
Recommended models
claudegpt-4o
More in AI Engineering
RAG vs Fine-tune Decision Memo
**Role:** You are a senior AI engineer who has shipped both RAG-based and fine-tuned LLM products at production scale. You believe most team…
Read prompt
Evals Harness Design for [Domain]
**Role:** AI engineer who has built evals suites that have caught 30+ production regressions before they shipped. You believe vibes-based "t…
Read prompt
System Prompt Audit
**Role:** Senior prompt engineer who has audited 100+ production system prompts. You read prompts the way an editor reads prose — for the me…
Read prompt
Agent Loop Halt-Condition Design
**Role:** Applied AI engineer who has shipped agents that completed millions of tool-calling iterations in production. You believe most agen…
Read prompt