Index: The Book of Statistical ProofsGeneral TheoremsFrequentist statisticsHypothesis testing ▷ Minimum detectable effect

Definition: Consider a hypothesis test concerning parameter $\theta$. We have $H_0: \theta \in \Theta_0$ versus $H_1: \theta \in \Theta_1$, where ${\Theta_0, \Theta_1}$ is a partition of $\Theta$. Let the probability of rejecting the null hypothesis for the test with $n$ observations be $\kappa_n(\theta)$. The number of samples $n$ is fixed.

The effect size for a parameter $\theta \in \Theta_1$ is the distance of $\theta$ from the null parameter space $\Theta_0, d(\theta, \Theta_0)$. It can be defined as

\[\label{eq:distancefromTheta0} d(\theta, \Theta_0) = \inf_{\theta_0 \in \Theta_0} ||\theta - \theta_0|| \; .\]

Let the test have a given significance level $\alpha$ and desired power $1 - \beta$. The minimum detectable effect, $\delta$, is the smallest $\delta$ so that the hypothesis test simultaneously has size less than or equal to $\alpha$ and power at least $1 - \beta$:

\[\label{eq:mdeconditions} \left(\sup_{\theta \in \Theta_0} \kappa_n(\theta) \leq \alpha\right) \wedge \left(\inf_{\theta: \, d(\theta, \Theta_0) \leq \delta} \kappa_n(\theta) \geq 1 - \beta\right) \; .\]
 
Sources:

Metadata: ID: D224 | shortcut: mde | author: alexanderdbolton | date: 2025-07-15, 00:00.