A/B Test Statistical Significance Calculator
Test whether your A/B test results are statistically significant. Enter visitors, conversions for control and variant to get the Z-score and p-value.
Calculate the relative and absolute uplift between control and variant in an A/B test. See the percentage improvement and confidence in the measured lift.
What if the measured uplift scales by different multipliers?
| Multiplier | Variant Value | Relative Uplift | Annual Revenue Impact |
|---|---|---|---|
| 0.5× | 3.23% | +7.50% | +$27,000.00 |
| 1× | 3.45% | +15.00% | +$54,000.00 |
| 2× | 3.9% | +30.00% | +$108,000.00 |
| 3× | 4.35% | +45.00% | +$162,000.00 |
| 5× | 5.25% | +75.00% | +$270,000.00 |
| Scenario | Recommendation |
|---|---|
| Significant positive uplift | ✅ Ship the variant |
| Significant negative uplift | ❌ Keep control, investigate variant |
| Not significant, small effect | ⏳ Run longer or increase traffic |
| Not significant, large effect | ⏳ Need more data — promising trend |
Uplift (or lift) measures how much better a variant performs compared to the control in an A/B test. While statistical significance tells you whether the difference is real, uplift tells you how big it is — the magnitude of the improvement.
This calculator computes both absolute uplift (the raw difference in conversion rates) and relative uplift (the percentage improvement over the control). Both metrics are essential: absolute uplift shows the impact in percentage points, while relative uplift shows the proportional improvement.
For example, a conversion rate going from 2% to 2.5% has an absolute uplift of 0.5 percentage points but a relative uplift of 25%. Both perspectives matter for different decisions. Absolute uplift maps directly to revenue; relative uplift contextualizes the size of the improvement.
Reporting only p-values leaves out the most important question: "How much better is it?" It gives the magnitude of improvement in both absolute and relative terms, giving you Complete View of your A/B test results.
Absolute Uplift = Variant CR − Control CR
Relative Uplift = (Variant CR − Control CR) / Control CR × 100
Uplift Direction: Positive = improvement, Negative = degradationResult: +15.0% relative uplift (+0.45pp absolute)
The variant converts at 3.45% vs. the control's 3.00%. Absolute uplift is 0.45 percentage points. Relative uplift is 0.45/3.0 × 100 = 15.0%. This is a meaningful improvement that would likely pass significance with adequate sample.
Typical e-commerce A/B test uplifts range from 1–5% for subtle changes (button color, copy tweaks) to 10–25% for structural changes (checkout flow redesign, pricing strategy). Revolutionary changes (new product recommendation engine) can reach 30–50% but are rare.
Sequential test wins compound multiplicatively. Three consecutive 5% wins produce (1.05)³ = 15.8% total uplift, not 15%. This compounding effect means consistent small wins create outsized long-term value. A CRO program delivering four 5% wins per year produces 21.6% annual improvement.
Convert uplift to revenue by multiplying absolute uplift by traffic and AOV. A 0.5 percentage point uplift on 100,000 monthly visitors at $80 AOV = 500 incremental orders = $40,000/month. This translation is essential for communicating CRO value to business stakeholders.
Last updated:
Absolute uplift is the raw difference in rates (3.45% − 3.0% = 0.45 percentage points). Relative uplift is the percentage change (0.45/3.0 = 15%). Both are useful: absolute maps to revenue, relative indicates proportional improvement.
Report both. Relative uplift contextualizes the improvement size. Absolute uplift connects directly to business impact. Stakeholders generally find relative uplift more intuitive but absolute uplift more actionable for revenue modeling.
A 10–15% relative uplift is a strong A/B test result. Most winning tests show 3–10% relative uplift. Anything above 20% is exceptional and should be verified carefully to rule out test contamination or novelty effects.
The MDE is the threshold you designed the test to detect. The actual observed uplift can be larger or smaller. If the observed uplift matches or exceeds the MDE and is significant, you have a clear winner.
Tests that barely pass significance tend to show smaller effects when replicated. The observed uplift at the significance boundary is statistically likely to be inflated by 10–30%. Discount accordingly for projections.
Yes. Negative uplift means the variant performed worse than the control. A significantly negative result is just as valuable as a positive one — it tells you what to avoid. Ship negative results just as rigorously as positive ones (don't implement the losing variant).
Test whether your A/B test results are statistically significant. Enter visitors, conversions for control and variant to get the Z-score and p-value.
Estimate the annualized revenue impact of an A/B test winner. Project how a conversion rate lift translates to additional dollars over time.
Calculate the confidence interval for a conversion rate or proportion. Support for 90%, 95%, and 99% confidence levels with sample size inputs.