AP Statistics · Unit 3 · Topic 3.8

Potential Errors When Performing Tests Study Lab

Every test decision can be wrong. A Type I error rejects a true null hypothesis; a Type II error fails to reject a false one. Power is the chance the test catches a real effect. See all three as areas and learn what moves them.

1. Errors and power explorer

A test can be wrong in two ways. The gray curve shows how p̂ varies if H₀ is true; the blue curve shows how p̂ varies if the true proportion is the value you set. The red dashed line is the cutoff: p̂ values beyond it lead us to reject H₀.

P(Type I error) = α P(Type II error) = β Power = 1 − β
Preset situations

In real life you never know this
Alternative
Significance level

Two sampling distributions

α (Type I risk)β (Type II risk)Power
P(Type I error) = α
P(Type II error) = β
Power = 1 − β
Reject H₀ if p̂ is…

Show the work

The two errors in this context

2. Key ideas

Type I

False alarm

Rejecting H₀ when it is true. Its probability is α, chosen before the test.

Type II

Missed effect

Failing to reject H₀ when it is false. Its probability is β, which depends on the true value of p.

Power

Catching a real effect

Power = 1 − β = P(reject H₀ when a specific alternative is true). Researchers usually want power of at least 0.80.

Trade-off

α vs. β

For a fixed n, lowering α makes Type I errors rarer but Type II errors more likely. Only a larger sample lowers both.

What raises power

Four levers

Larger n, larger α, a true value farther from p₀ (bigger effect), and less variability.

Which is worse?

It depends on context

Ask what happens after each mistake. If a false alarm is costly, choose a small α; if missing an effect is costly, aim for high power.

3. Quick reference

The four possible outcomes of a test.

DecisionIf H₀ is trueIf H₀ is false
Reject H₀Type I error, probability αCorrect decision, probability = power
Fail to reject H₀Correct decision, probability 1 − αType II error, probability β

4. On the FRQ

How to earn the points

  • Describe an error in context with both parts: what the truth is and what you concluded. "A Type I error would be concluding that more than 5% of packages are late when in fact 5% are late."
  • Give a consequence of the error in context when asked.
  • If the decision was "reject H₀," only a Type I error is possible; if "fail to reject," only Type II.
  • To increase power without raising the Type I error rate, say "increase the sample size."
  • Interpret power: "If the true late rate is 8%, there is about a 0.81 probability the test (with n = 400 and α = 0.05) will find convincing evidence that the rate is above 5%."

Common mistakes

  • Defining errors without context ("rejecting a true null" alone may not earn credit).
  • Saying α is the probability that H₀ is true, or that the decision is wrong.
  • Thinking a larger α lowers power. A larger α makes it easier to reject, so power goes up.
  • Saying both error probabilities can be lowered by changing α. Only a bigger sample (or less variability) lowers both.
  • Mixing up which error is possible given the test's actual decision.

5. Practice questions

AP-style multiple choice. Pick an answer to see the explanation.

0 / 0 correct

6. Key terms

The vocabulary for topic 3.8. Read the list, flip through flashcards, or quiz yourself. For every term in the unit, see the Unit 3 terminology guide.