Today’s Agenda

  • Power
  • Example

Decision Errors

\(H_0\) is True \(H_0\) is False
Do not reject \(H_0\) Correct conclusion Type II error
Reject \(H_0\) Type I error Correct conclusion
  • Assuming the null hypothesis true, the probability of making a type I error is the significance level \(\alpha\)
  • the probability of not rejecting the null hypothesis when it is actually false is often called \(\beta\)
  • the probability of correctly rejecting the null hypothesis in the case that is false is called the power of our test

Example: The researchers would like to run the clinical trial on patients with systolic blood pressures between \(140\) and \(180\) mmHg. Suppose previously published studies suggest that the standard deviation of the patients’ blood pressures will be about \(12\) mmHg and the distribution of patient blood pressures will be approximately normal. Out of \(150\) patients, \(75\) are given a new medication meant to reduce blood pressure and the other \(75\) are given the standard treatment. Set up hypotheses to determine if the medicine reduces the average blood pressure of the group taking the new medication.

\(H_0: \mu_{new} - \mu_{old} \geq 0\)

\(H_A: \mu_{new} - \mu_{old} < 0\)

  1. Is this paired data? Not paired data, there are not people in the two groups with a special connection
# find cut-off value for our significance level
se <- sqrt(12^2/75 + 12^2/75)
qnorm(.95)
## [1] 1.644854
0 - qnorm(.95)*se
## [1] -3.223242
  • If we reject the null hypothesis we would say “our data shows statistically significant evidence the the medication lower blood pressure in patients”
  • However, this does not mean the medication is worth it
  • Potentially there will be negative side effects to the new medicine or be incredibly expensive
  • Researchers determine that the medicine will only be worth it if it reduces blood pressure by at least \(3\) mmHg

So, we want to know, “how likely are we to notice an effect of a \(3\) mmHg reduction?”.

  1. If there truly is a \(3\) mmHg reduction but we fail to reject the null hypothesis, what type of an error is this?

Type II error

  • Imagine now a world where there really is a \(3\) mmHg decrease in blood pressure, draw the sampling distribution in this reality.

Draw a picture with two potential sampling distributions: the sampling distribution where we assume the null hypothesis is true and the the sampling distribution where we assume the effect of \(3\) mmHg reduction is true. Label the area that represents the probability of correctly rejecting the null hypothesis assuming it is wrong. (Assume a \(5 \%\) significance level)

  1. Find the power of the test at detecting a \(3\) mmHg reduction in blood pressure. (This is one minus the shaded area above)
# find z-score of cut-off value
z <- (-3.22 - (-3))/se
pnorm(z)
## [1] 0.4553053
# our test has 45% power
# if the null hypothesis is actually false, and the medication DOES cause a 3mmHg reduction in blood pressure, there is only a 45% chance that our hypothesis test will reject the null hypothesis.
# we often want tests to have at least 80% power
  1. Suppose we do a random sample of \(15\) values from a normally distributed population where \(\sigma =6\) and \(\mu\) is unknown. We want to perform a hypothesis test using a significance level \(\alpha = .05\) where the hypotheses are given below. What is the power of the test at detecting a mean of \(54\)? Include a picture with two potential sampling distributions: the sampling distribution where we assume the null hypothesis is true and the the sampling distribution where we assume the true mean is \(54\) and shade the region representing the power.

\(H_0: \mu = 50\)

\(H_A: \mu > 50\)

# find cut-off
se <- 6/sqrt(15)
50 + qnorm(.95)*se
## [1] 52.5482
# find z-score
z <- (52.55-54)/se
1-pnorm(z)
## [1] 0.8253559
#This test has a power of 82.5% at detecting a true mean of 54
# 82.5% of the time, in the hypothesis test that we set up above, the null hypothesis will correctly be rejected if the true mean is 54
  1. Suppose we do a random sample of \(42\) values from a normally distributed population where \(\sigma =16\) and \(\mu\) is unknown. We want to perform a hypothesis test using a significance level \(\alpha = .10\) where the hypotheses are given below. What is the power of the test at detecting a mean of \(275\)? Include a picture with two potential sampling distributions: the sampling distribution where we assume the null hypothesis is true and the the sampling distribution where we assume the true mean is \(275\) and shade the region representing the power.

\(H_0: \mu = 300\)

\(H_A: \mu < 300\)

  1. Suppose that we randomly sample scores on a test from two groups; \(315\) observations from group \(A\) and \(225\) observations from group \(B\). Group \(A\) has population standard deviation \(32\) and group \(B\) has population standard deviation \(25\). We want to perform a hypothesis test using a significance level \(\alpha = .10\) where the hypotheses are given below. What is the power of the test at detecting a group \(A\) having an average of \(20\) higher than group \(B\) ? Include a picture with two potential sampling distributions: the sampling distribution where we assume the null hypothesis is true and the the sampling distribution where we assume the group \(A\) scores \(20\) points higher and shade the region representing the power.

\(H_0: \mu_A - \mu_B \leq 0\)

\(H_A: \mu_A - \mu_B > 0\)