Statistical Hypothesis Test Sensitivity

A Statistical Hypothesis Test Sensitivity is a probability value that a statistical hypothesis test correctly rejects a false null hypothesis.

• AKA: Statistical Power.
• Context:
  • It can be expressed as the conditional probability function, [math]\displaystyle{ P\big( \mbox{reject } H_0 \big| H_1 \mbox{ is true} \big), }[/math] where [math]\displaystyle{ H_0 }[/math] is a null hypothesis and [math]\displaystyle{ H_1 }[/math]an alternative hypothesis.
  • It can also be defined as [math]\displaystyle{ 1 - \beta }[/math] where [math]\displaystyle{ \beta }[/math] is the rate of a type II error.
• Example(s):
  • Considering a random sample of N=64 cases is drawn from a statistical population with true mean [math]\displaystyle{ \mu_X=86.5 }[/math] and standard deviation, [math]\displaystyle{ \sigma_X=8 }[/math]. The statistical power of the following hypothesis test

• [math]\displaystyle{ H_0 }[/math]: the distribution of all possible means for samples of size N has a mean equal to the observed sample mean, i.e. [math]\displaystyle{ \mu_{\bar{X}} = 85 }[/math]" (null hypothesis)
• [math]\displaystyle{ H_1 }[/math]: the observed sample mean falls in the rejection region, [math]\displaystyle{ \mu_{\bar{X}} \ge 86 }[/math]" (alternative hypothesis).

is estimated bo be

[math]\displaystyle{ P\big( \mbox{reject } H_0 \big| H_1 \mbox{ is true} \big)=P(\bar{X} \ge 86 \text { if } \mu = 86.5) = P(Z \ge -0.50) = 0.6915 }[/math]

where [math]\displaystyle{ Z=(\bar{X}-\mu_X)/\sigma_{\bar{X}} }[/math] with [math]\displaystyle{ \sigma_{\bar{X}}=\sigma_X/\sqrt{N}=8/\sqrt{64}=1 }[/math]. This means, there is a 69.15% chance of choosing the correct hypothesis. Alternatively, the statistical power is can also be calculated by [math]\displaystyle{ 1-\beta=1-0.3085= 0.6915 }[/math] where [math]\displaystyle{ \beta = P(\bar{X} \le 86 \text { if } \mu = 86.5) = P(Z \le -0.50)= 0.3085 }[/math].

• See: Nonparametric Test, Sensitivity And Specificity, Null Hypothesis, Type II Error, Type I Error, Sample Size, Effect Size, Parametric Statistics.

---

---

References

2015

• (Wikipedia, 2015) ⇒ http://en.wikipedia.org/wiki/statistical_power Retrieved:2015-9-8.
  • The power or sensitivity of a binary hypothesis test is the probability that the test correctly rejects the null hypothesis (H₀) when the alternative hypothesis (H₁) is true. It can be equivalently thought of as the probability of accepting the alternative hypothesis (H₁) when it is true – that is, the ability of a test to detect an effect, if the effect actually exists. That is, : [math]\displaystyle{ \mbox{power} = \mathbb P\big( \mbox{reject } H_0 \big| H_1 \mbox{ is true} \big) }[/math] The power of a test sometimes, less formally, refers to the probability of rejecting the null when it is not correct. Though this is not the formal definition stated above.

    The power is in general a function of the possible distributions, often determined by a parameter, under the alternative hypothesis. As the power increases, there are decreasing chances of a Type II error (false negative), which are also referred to as the false negative rate (β) since the power is equal to 1−β, again, under the alternative hypothesis. A similar concept is Type I error, also referred to as the "false positive rate" or the level of a test under the null hypothesis.

    Power analysis can be used to calculate the minimum sample size required so that one can be reasonably likely to detect an effect of a given size. Power analysis can also be used to calculate the minimum effect size that is likely to be detected in a study using a given sample size. In addition, the concept of power is used to make comparisons between different statistical testing procedures: for example, between a parametric and a nonparametric test of the same hypothesis.

2008

• (Park, 2008) ⇒ Hun Myoung Park (2008). “Hypothesis testing and statistical power of a test". The University Information Technology Services (UITS) Center for Statistical and Mathematical Computing, Indiana University, 102. URL: http://hdl.handle.net/2022/19738 , Full Report(PDF file)
  • What is the power of a test? The power of a statistical test is the probability that it will correctly lead to the rejection of a false null hypothesis (Greene, 2000). The statistical power is the ability of a test to detect an effect, if the effect actually exists (High, 2000). Cohen (1988) says, it is the probability that it will result in the conclusion that the phenomenon exists (p.4). A statistical power analysis is either retrospective (post hoc) or prospective (a priori). A prospective analysis is often used to determine a required sample size to achieve target statistical power, while a retrospective analysis computes the statistical power of a test given sample size and effect size.

1977

• (Cohen, 1977) ⇒ Jacob Cohen (1977). “Statistical power analysis for the behavioral sciences" (revised ed.) ISBN: 978-0-12-179060-8, pp. 53-56.
  • The statistical power of a significance test is the long-term probability, given the population ES, [math]\displaystyle{ \alpha }[/math], and N of rejecting [math]\displaystyle{ H0 }[/math]. When the ES is not equal to zero, [math]\displaystyle{ H_0 }[/math] is false, so failure to reject it also incurs an error. This is a Type II error, and for any given ES, [math]\displaystyle{ \alpha }[/math], and N, its probability of occurring is [math]\displaystyle{ \beta }[/math]. Power is thus [math]\displaystyle{ 1-\beta }[/math], the probability of rejecting a false [math]\displaystyle{ H_0 }[/math].

---

---