The same baseline with different minimum detectable effects produces dramatically different sample size requirements.
Example A: 15% MDE
Baseline conversion rate3.0%
Minimum detectable effect15% relative
Target conversion rate3.45%
Statistical power80%
Significance level5% (p < 0.05)
Required per variant
23,000 visitors
Duration at 2,000/day
23 days
Example B: 5% MDE
Baseline conversion rate3.0%
Minimum detectable effect5% relative
Target conversion rate3.15%
Statistical power80%
Significance level5% (p < 0.05)
Required per variant
210,000 visitors
Duration at 2,000/day
210 days
Key insight: Reducing the MDE from 15% to 5% increases the required sample size by nearly 10x. Detecting a 1% relative lift requires roughly 100x more data than detecting a 10% lift. The MDE is a business decision, not a statistical one.