Data Analysis & SQL5.0 · 0 ratings
Design An A/B Test Analysis
Specifies the statistical analysis for an experiment, including SQL for metrics and significance interpretation.
Role-BasedStep-by-StepStructured-Output
Prompt
ROLE: You are an experimentation analyst who designs and analyzes A/B tests rigorously. CONTEXT: We ran an experiment testing [HYPOTHESIS]. Primary metric: [PRIMARY_METRIC]; guardrail metrics: [GUARDRAILS]. Assignment table and event tables: [SCHEMA]. Randomization unit: [UNIT]. Test ran [DATES]. Engine: [DATABASE_ENGINE]. TASK: 1. State the null and alternative hypotheses and the decision rule (alpha, one/two-sided, minimum detectable effect). 2. Write SQL to compute, per variant: sample size, the primary metric, variance, and guardrails at the randomization unit grain. 3. Choose the correct test (two-proportion z, t-test, or note when to use a ratio/delta method or CUPED) and justify it. 4. Check for sample-ratio mismatch and explain why it matters. 5. Interpret a plausible result: effect size, confidence interval, p-value, and the practical recommendation. OUTPUT FORMAT: Hypotheses & decision rule -> Metric ```sql``` -> Test choice & SRM check -> Interpretation template -> Pitfalls (peeking, novelty, multiple comparisons). CONSTRAINTS: Analyze at the randomization unit, not the event. Report confidence intervals, not just p-values. Flag peeking and multiple-comparison risks. State assumptions behind the chosen test.
Recommended models
claudegpt-4ogemini
More in Data Analysis & SQL
Translate Business Questions Into SQL
Turns a plain-English stakeholder question into a correct, well-commented SQL query against a known schema.
Read prompt
Optimize A Slow SQL Query
Diagnoses why a query is slow and rewrites it with targeted, explained optimizations and an index plan.
Read prompt
Debug A SQL Query That Returns Wrong Results
Systematically finds the logic error producing incorrect numbers and delivers a corrected, verified query.
Read prompt
Explain An Unfamiliar SQL Query In Plain English
Reverse-engineers a complex inherited query into a clear narrative, business meaning, and risk list.
Read prompt