How many visitors (or recipients) does each version need before you can trust the result? Enter your current rate and the smallest improvement worth detecting.
Visitors needed per version
–
Total visitors for the test (A + B)
–
Rate you are trying to detect
–
Share of your traffic used
–
Left for the winning version
–
What is the A/B Test Sample Size Calculator?
Most A/B tests are too small to prove anything. Before you start, find out how many visitors or recipients each version needs to detect the improvement you care about.
Enter your current conversion rate, the smallest lift worth detecting, and the confidence and power you want.
How to use the A/B Test Sample Size Calculator
- Enter your current conversion rate.
- Enter the smallest relative improvement you want to detect (e.g. 10%).
- Keep 95% confidence and 80% power unless you have a reason to change them.
- Enter the traffic you have to see the share the test needs.
Understanding your results
- Visitors per version
- How many people each version must receive.
- Share of your traffic
- If the test needs most of your traffic, test bolder changes instead.
Frequently asked questions
Why do small improvements need large samples?
Random variation between groups is large compared with small differences.
What is statistical power?
The chance of detecting a real improvement. 80% is the usual standard.
Can I stop a test early?
Stopping early when one version looks ahead produces false winners. Reach your sample size first.