Introducing Statistics:
Should I Worry About Error?
Learn to identify, measure, and evaluate sources of error in statistical studies: from sampling variability to bias, and understand when error truly matters.
What Is Error in Statistics?
In statistics, error is the difference between what we observe in a sample and the true value of the population parameter. Every study has some error: the question is whether it is small enough to trust our conclusions.
Why Should You Care About Error?
Imagine a poll says 52% of voters support a candidate, but the margin of error is \(\pm 4\%\). The true support could be anywhere from 48% to 56%: the race could go either way. Understanding error tells us how much to trust a result.
Two Families of Error
All statistical error falls into two broad categories. Understanding the difference is essential for designing good studies and interpreting results.
Bias vs. Variability: The Bullseye
The Margin of Error
The margin of error quantifies the expected range of sampling variability. For a proportion at the 95% confidence level:
How Sample Size Reduces Error
Because the margin of error has \( \sqrt{n} \) in the denominator, quadrupling the sample size only halves the margin of error. This is the law of diminishing returns.
Bias vs. Variability: Key Distinction
| Sampling Variability | Bias | |
|---|---|---|
| Type | Random error | Systematic error |
| Direction | Equally likely above or below | Consistently one direction |
| Effect of larger \( n \) | Decreases | No change |
| Fix | Increase sample size | Improve study design |
| Detected by | Repeated sampling | Comparison to known values |
Common Sources of Bias
How to Evaluate Error
Example 1: The School Poll
A school newspaper surveys 100 students and finds that 60% support a new dress code. Compute the margin of error at 95% confidence.
Example 2: The National Survey
A national poll of \( n = 1{,}500 \) adults finds 47% approve of a policy. The poll uses random digit dialing. Should we worry about error?
Example 3: The Website Survey
A website posts an open poll asking "Do you support the new tax?" and gets 10,000 responses. The result: 78% say no. The site claims "Americans overwhelmingly oppose the tax."
When Does Error Actually Matter?
Error matters most when the conclusion could flip depending on the true value. Ask: does the confidence interval include values that would change my decision?
Common Mistakes
Summary: Error at a Glance
| Sampling Variability | Bias | |
|---|---|---|
| Nature | Random fluctuation | Systematic shift |
| Measured by | Margin of error / SE | Study design review |
| Reduced by | Increasing \( n \) | Better sampling methods |
| Worry when | CI is wide relative to claim | Non-random or voluntary sample |
| Formula captures it? | Yes | No |
Question 1 of 5
A researcher surveys 2,000 randomly selected adults and finds that 62% support a new policy. The margin of error is \(\pm 2.1\%\). Which statement is most accurate?
Question 2 of 5
A website posts an online poll and receives 50,000 responses. The margin of error is tiny: \(\pm 0.4\%\). Why should we still worry about this result?
Question 3 of 5
If a researcher wants to cut the margin of error in half, by what factor must they increase the sample size?
Question 4 of 5
A poll finds that 51% of voters favor Candidate A with a margin of error of \(\pm 3\%\). What is the best interpretation?
Question 5 of 5
Which of the following is a source of bias rather than sampling variability?