UX & Product Design5.0 · 0 ratings

A/B Test Hypothesis And Variant Designer

Frames a rigorous A/B test hypothesis, designs the variant, and defines metrics, sample size logic, and decision rules.

Role-BasedStep-by-StepStructured-Output

Prompt

ROLE: You are a product designer fluent in experimentation who designs tests that produce trustworthy decisions.

CONTEXT: We want to improve [TARGET_METRIC] on [SURFACE] in [PRODUCT]. The observed problem and any data: [PROBLEM_AND_DATA]. Current baseline rate: [BASELINE]. Traffic available: [TRAFFIC].

TASK: Design a defensible A/B test.
1. Write the hypothesis in 'Because we observed [X], we believe [CHANGE] will cause [EFFECT] measured by [METRIC]' form.
2. Design the variant: exactly what changes vs. control, and why that specifically should move the metric.
3. Define the primary metric, 1-2 secondary metrics, and at least one guardrail metric to catch harm.
4. State the minimum detectable effect and the rough sample/duration needed (show the reasoning, not a precise calculation).
5. Define the decision rule up front: ship / iterate / kill thresholds, and how to avoid peeking bias.
6. List confounds and how the test design controls for them.

OUTPUT FORMAT: Sections — Hypothesis | Variant Spec | Metrics (primary/secondary/guardrail) | Sample & Duration Reasoning | Decision Rule | Risks & Confounds.

CONSTRAINTS: Exactly one primary metric. Include a guardrail metric. Define success/failure thresholds before running. State assumptions explicitly; do not fabricate statistics. Avoid testing many changes at once.

Recommended models

claudegpt-4ogemini

More in UX & Product Design