# A Guide to Calculating Sample Size and Experiment Length with Minimum Detectable Effect (MDE)

In the fast-paced world of technology, staying ahead of the competition requires continuous innovation and informed decision-making. Tech companies have realized the immense potential of experimentation as a powerful tool to validate ideas, measure the impact of changes, and make data-driven decisions.

AB testing is a technique used in digital marketing and product development to compare multiple variations of a webpage or app screen. By dividing the audience into different groups and measuring user behavior, organizations can determine which variation performs better.

In the world of experimentation (in this case is A/B Testing), determining the appropriate sample size and experiment length is crucial for obtaining reliable and statistically significant results. One key factor in this process is the Minimum Detectable Effect (MDE). In this article, we will explore how to calculate the sample size and experiment length based on the desired MDE, ensuring accurate and meaningful experimentation.

Understanding Minimum Detectable Effect (MDE)

The Minimum Detectable Effect represents the smallest change or difference in your experiment metric that you want to detect as statistically significant. It is essential to define the MDE upfront to ensure your experiment is adequately powered to detect meaningful effects.

Sample Size Calculation

To calculate the required sample size, you need to consider several factors:

a. Confidence Level: Choose a confidence level that represents your desired level of certainty, typically set at 95%.

b. Power: Determine the power of the experiment, which represents the probability of detecting an effect if it truly exists. A commonly used value is 80%.

c. Baseline Conversion Rate: Determine the current conversion rate or performance metric of your control group.

d. MDE: Define the desired Minimum Detectable Effect, which is the smallest change you want to detect.

Experiment Length Calculation

Once you have determined the sample size, you can estimate the experiment length. Consider the following factors:

Expected Traffic or User Volume: Assess the expected number of visitors or users during the experiment period.

Conversion Rate Stability: Evaluate the stability of your conversion rate over time.

Sample Allocation: Determine the percentage of users allocated to each variant in your experiment. d. Statistical Significance Criteria: Choose the statistical significance threshold to determine when the experiment can be concluded.

Online Tools and Calculators

Utilize online sample size calculators and statistical tools designed specifically for A/B testing and experimentation. These tools simplify the calculation process by incorporating the necessary statistical formulas and considerations.

Real Case