01 / 21
p

Interpreting
p-Values

Learn how to correctly interpret the p-value in hypothesis testing: what it measures, what it does not mean, and how to use it to make statistical decisions.

AP Statistics

What Is a p-Value?

A p-value is the probability of obtaining a test statistic at least as extreme as the one observed, assuming the null hypothesis is true.

Think of the p-value as a measure of surprise: how surprising is our observed result if nothing interesting is really going on (i.e., the null is correct)?

Visualizing the p-Value

Test Statistic Values observed p-value (shaded area)
Distribution under \( H_0 \)

Why p-Values Matter

Evidence Strength
A smaller p-value means stronger evidence against the null hypothesis: the observed data would be very unlikely if \( H_0 \) were true.
Decision Framework
The p-value is compared against a pre-set significance level \( \alpha \) (often 0.05) to decide whether to reject \( H_0 \).
Universal Language
p-values provide a standardized way for researchers across all fields to communicate the strength of their statistical evidence.

The Decision Rule

Reject \( H_0 \)
\( p \le \alpha \)
Statistically significant
Fail to Reject \( H_0 \)
\( p > \alpha \)
Not statistically significant

Formal Definition

Right-tailed
\( p = P(T \geq t_{\text{obs}} \mid H_0) \)
Left-tailed
\( p = P(T \leq t_{\text{obs}} \mid H_0) \)
Two-tailed
\( p = 2 \cdot P(T \geq |t_{\text{obs}}| \mid H_0) \)

The direction of the test (one-sided vs. two-sided) determines which tail(s) of the sampling distribution contribute to the p-value.

One-Tailed vs. Two-Tailed Tests

One-Tailed (Right)
\(p\)
Two-Tailed
\(\frac{p}{2}\)
\(\frac{p}{2}\)
In a two-tailed test, the p-value includes both extreme tails. So a two-tailed p-value is twice the one-tailed area.

What p-Values Are and Are Not

✓ p-Value IS ✕ p-Value IS NOT
Probability of... ...data this extreme given \( H_0 \) true ...\( H_0 \) being true
Measures Compatibility of data with \( H_0 \) Size or importance of an effect
Tells you How surprising the result is under \( H_0 \) Whether you should believe \( H_a \)
Depends on Sample size, effect, and variability Only the truth of \( H_0 \)

The Significance Level \( \alpha \)

The significance level \( \alpha \) is a threshold chosen before collecting data. It represents the maximum acceptable probability of a Type I error: rejecting \( H_0 \) when it is actually true.

\( \alpha = 0.05 \)
Most common in social sciences and AP Statistics. Willing to accept a 5% chance of a false positive.
\( \alpha = 0.01 \)
More stringent: used in medical research or high-stakes decisions. Only 1% risk of Type I error.

How to Interpret a p-Value

1
State the hypotheses. Write \( H_0 \) and \( H_a \) clearly before any calculation.
2
Choose \( \alpha \). Set your significance level (typically \( \alpha = 0.05 \)) before collecting data.
3
Compute the test statistic and p-value. Use the appropriate test (z, t, \( \chi^2 \)) to find the p-value.
4
Compare p-value to \( \alpha \). If \( p \leq \alpha \), reject \( H_0 \). If \( p > \alpha \), fail to reject \( H_0 \).
5
Write a conclusion in context. State whether there is convincing evidence for \( H_a \), referencing the specific context of the problem.

Example 1: Coin Fairness

A coin is flipped 100 times and lands heads 62 times. Test whether the coin is fair using \( \alpha = 0.05 \).

Example 2: Battery Life

A manufacturer claims batteries last \( \mu = 500 \) hours on average. A sample of \( n = 36 \) batteries has \( \bar{x} = 490 \) hours and \( s = 30 \) hours. Test at \( \alpha = 0.05 \).

Example 3: Fail to Reject

A school claims that 70% of students pass a skills test. In a random sample of 200 students, 134 pass. Test at \( \alpha = 0.05 \).

Writing AP-Style Conclusions

On the AP exam, your conclusion must include four elements. Missing any one can cost you points.

1
Compare: "Since the p-value of ___ is less than (or greater than) \( \alpha = \) ___..."
2
Decision: "...we reject (or fail to reject) \( H_0 \)."
3
Evidence statement: "There is (or is not) convincing evidence that..."
4
Context: Reference the specific population, parameter, and claim from the problem.

Common Mistakes

✕ "\( p = \) probability \( H_0 \) is true"
The p-value is not the probability that \( H_0 \) is true. It assumes \( H_0 \) is true and measures how surprising the data are.
✕ "We accept \( H_0 \)"
We never "accept" \( H_0 \). We either reject it or fail to reject it. Absence of evidence is not evidence of absence.
✕ "Small \( p \) = large effect"
A very small p-value does not mean the effect is large or important. With a huge sample, even a tiny difference can yield a small p.

Summary Reference Table

p-Value Range Evidence Against \( H_0 \) Typical Action
\( p > 0.10 \) Weak or none Fail to reject \( H_0 \)
\( 0.05 < p \le 0.10 \) Moderate (marginal) Fail to reject at \( \alpha = 0.05 \)
\( 0.01 < p \le 0.05 \) Strong Reject at \( \alpha = 0.05 \)
\( p \le 0.01 \) Very strong Reject at \( \alpha = 0.01 \)

Question 1 of 5

A researcher obtains a p-value of 0.03 when testing \( H_0: \mu = 100 \) vs. \( H_a: \mu \neq 100 \) at \( \alpha = 0.05 \). Which conclusion is correct?

Question 2 of 5

A test of \( H_0: p = 0.5 \) vs. \( H_a: p > 0.5 \) yields a p-value of 0.12. At \( \alpha = 0.05 \), what should you conclude?

Question 3 of 5

Which of the following is the best interpretation of a p-value of 0.04?

Question 4 of 5

A study with \( n = 10{,}000 \) finds a statistically significant result with \( p = 0.001 \). Which statement is most accurate?

Question 5 of 5

A two-tailed test has a p-value of 0.08. If the same data were used in a one-tailed test (in the correct direction), the one-tailed p-value would be approximately: