AI Engineering5.0 · 50 ratings

Vendor LLM Evaluation

**Role:** AI vendor selection lead. **Context:** Team evaluating LLM providers (OpenAI / Anthropic / Google / Mistral / self-hosted). Need …

Role-BasedChain-of-Thought

Prompt

**Role:** AI vendor selection lead.

**Context:** Team evaluating LLM providers (OpenAI / Anthropic / Google / Mistral / self-hosted). Need rigorous comparison.

**Task:** Eval:
1. Eval criteria: capability, cost, latency, compliance, vendor lock-in, SLA, support, roadmap.
2. Per-vendor scoring across each dimension.
3. Capability benchmark on YOUR use case (not generic MMLU).
4. Cost projection for YOUR scale.
5. Compliance: SOC 2, HIPAA, EU residency, indemnification.
6. Switch cost: how to migrate off this vendor later.
7. Recommendation + backup.
8. Re-evaluation schedule.

**Constraints:**
- Use-case-specific benchmark, not industry-standard.
- Cost is your real projected cost, not list price.

**Output format:** Vendor comparison table + recommendation + migration plan.

How to use this prompt

  1. 1

    Copy the prompt above and paste it into ChatGPT, Claude, or Gemini — or open it in the visual Studio to edit each part on a canvas and run it with your own key.

  2. 2

    Replace any bracketed placeholders with your specifics. The more concrete your context and constraints, the sharper the result — see the 5-part prompt structure.

  3. 3

    Run it, then refine. Ask the model to critique and improve its own answer with self-critique prompting.

Techniques in this prompt

Role-Based

Assigns the model an expert persona so it adopts the right vocabulary, depth, and standards for the task.

Learn this technique
Chain-of-Thought

Asks the model to reason step by step before answering — ideal for multi-step, logical, or analytical tasks.

Learn this technique

Recommended models

claudegpt-4o

Build on this prompt

Open it in the visual Studio to wire it into a full workflow with your own API key — or learn the craft behind prompts like this.

More in AI Engineering