A/B Price Test Sample Size Calculator

About the A/B Price Test Sample Size Calculator

Running a price test without enough traffic is like flipping a coin and calling it science. Our A/B Price Test Sample Size Calculator tells you exactly how many visitors or transactions each variant needs before you can trust the results. Enter your baseline conversion rate, the minimum change you want to detect, and your desired confidence and power levels — the tool returns a clear sample size per group along with estimated test duration.

Price experiments carry more risk than typical UX tests because every transaction affects real revenue. Under-powered tests lead to false positives that can lock in a worse price, while over-powered tests waste time that could be spent on other optimizations. This calculator uses the standard two-proportion z-test formula so you can plan experiments that are both efficient and reliable.

Whether you are testing a small SaaS pricing change or a major e-commerce markdown strategy, knowing the required sample size upfront prevents premature conclusions and protects your bottom line.

When This Page Helps

Guessing when a price test has “enough data” is the most common mistake in pricing optimization. Ending an experiment too early often means acting on statistical noise, while running too long wastes traffic that could power the next test. This calculator removes the guesswork by applying established statistical formulas, helping you allocate resources efficiently and reach trustworthy conclusions every time.

How to Use the Inputs

1. Enter your current (control) conversion rate as a percentage.
2. Specify the minimum detectable effect — the smallest improvement worth detecting.
3. Choose a significance level (commonly 5% for 95% confidence).
4. Choose statistical power (commonly 80%).
5. Optionally enter your daily traffic to estimate test duration.
6. Read the required sample size per variant and total.
7. Use the scenario table to compare different MDE and confidence combinations.

Example Calculation

Tips & Best Practices

• A 0.5 pp MDE on a 3% baseline means detecting a 17% relative lift — set MDE based on business impact, not just what's easy to detect.
• Raising confidence from 90% to 99% roughly doubles the required sample size.
• If you can't reach the sample size in a reasonable time, consider a larger MDE or sequential testing methods.
• Always calculate sample size BEFORE starting the test, not after.
• Run the test for full weeks to avoid day-of-week bias in purchase behavior.
• Segment analysis (mobile vs. desktop) requires even larger samples — each segment needs its own power.

Why Sample Size Matters for Price Tests

Pricing is one of the highest-leverage decisions a business can make. A well-run price test can reveal whether a higher price boosts revenue without losing conversions, or whether a lower price drives enough volume to compensate for thinner margins. But these insights are only useful if you can trust the data. Under-sized experiments produce noisy results that look convincing but don't replicate, leading to pricing mistakes that can persist for months or years.

Fixed-Horizon vs. Sequential Testing

This calculator uses the fixed-horizon approach: compute a sample size upfront, run the test until you hit it, then analyze. The main alternative is sequential testing (e.g., SPRT), which allows early stopping when results are conclusive. Sequential methods are powerful but require more statistical sophistication and infrastructure. For most teams, a fixed-horizon test with a pre-computed sample size is the simplest reliable approach.

Practical Tips for Pricing Experiments

Always randomize at the user level, not the session level, so returning visitors see a consistent price. Run the test for complete weeks to neutralize day-of-week effects. Log revenue and conversion data separately so you can analyze both. And document your hypothesis, MDE, and sample size plan before launching — post-hoc rationalizations are the enemy of rigorous experimentation.

Frequently Asked Questions

• What is minimum detectable effect (MDE)?

  MDE is the smallest difference between control and variant conversion rates that your experiment can reliably pick up. A smaller MDE requires more traffic. Choose an MDE that represents a meaningful business impact — a 0.1 pp lift on a 3% baseline may not justify the engineering cost of the change.

• What happens if I end the test early?

  Stopping an experiment before reaching the required sample size inflates your false positive rate. You might conclude a price change works when it actually doesn't, or miss a real improvement. The calculated sample size assumes a fixed-horizon test — peeking at results and stopping early violates that assumption.

• Should I use 80% or 90% power?

  80% power is the standard in most industries and balances sample size with reliability. It means there is a 20% chance you fail to detect a real effect. For high-stakes pricing decisions where a miss is costly, 90% power provides extra protection at the expense of roughly 30% more traffic.

• How do I handle a revenue-based metric instead of conversion rate?

  Revenue metrics have higher variance than binary conversion rates, so they require larger samples. This calculator focuses on conversion rate (binary outcome). For revenue per visitor, you would use a t-test formula with the standard deviation of revenue, which typically requires 2–5× more traffic than a conversion test.

• Can I test more than two prices at once?

  Yes, but each additional variant increases the total sample needed. A three-variant test requires pairwise comparisons and a correction like Bonferroni to control the overall false positive rate. As a rule of thumb, multiply the two-variant sample by 1.5 for three variants.

• What significance level should I use?

  The standard is 5% (95% confidence), meaning there's a 5% chance of a false positive. For pricing tests with large revenue impact, some teams use 1% (99% confidence) for extra safety. Lower significance requires larger samples, so there's always a trade-off.