Sampling Distributions — I can use simulation to estimate a...

SECTION 1

Historical Context & Motivation

Imagine you want to know the average height of every student in your school, but you only have time to measure 30 people. How confident should you be that those 30 heights tell you something useful about the whole school? This question — how well a sample represents a population — has driven statisticians for centuries. The key idea that unlocks the answer is the sampling distribution, which shows what happens when you repeat the sampling process many, many times.

1733

De Moivre's Bell Curve

Abraham de Moivre discovered that repeated coin-flip experiments produce results that cluster into a bell-shaped curve, planting the seed for understanding how sample results naturally vary.

1809

Gauss & the Normal Distribution

Carl Friedrich Gauss formalized the normal distribution while studying measurement errors in astronomy, showing that averages of many observations tend to form a predictable pattern.

1908

Gosset's t-Distribution

William Sealy Gosset (writing under the pseudonym 'Student') developed the t-distribution while working at a brewery, solving the problem of small sample sizes in quality control.

1930s–1940s

Formal Sampling Theory

Jerzy Neyman and Egon Pearson built the mathematical framework for sampling distributions, confidence intervals, and hypothesis testing that forms the backbone of modern statistics.

1990s–Present

Computer Simulation Era

With personal computers, anyone can now simulate thousands of samples in seconds, making the concept of a sampling distribution tangible rather than purely theoretical.

The big question that motivated all of this work is deceptively simple: If I take a random sample and calculate a statistic (like the mean), how much will that statistic bounce around from sample to sample? Understanding this variability is the foundation of every poll you see on the news, every clinical drug trial, and every A/B test run by tech companies.

SECTION 2

Core Principles & Definitions

Before we dive into simulations, let's lock down four foundational ideas. Each one builds on the last, so take them in order.

Population vs. Sample

The population is the entire group you care about (e.g., all 1,200 students). A sample is a smaller subset you actually measure (e.g., 30 students chosen at random).

Parameter vs. Statistic

A parameter is a fixed number describing the population (like the true mean μ). A statistic is a number calculated from a sample (like the sample mean x̄). Statistics change from sample to sample; parameters don't.

Sampling Variability

Every random sample produces a slightly different statistic. This natural fluctuation is called sampling variability. It's not an error — it's an expected feature of randomness.

Sampling Distribution

If you could take every possible sample of the same size and record the statistic each time, the distribution of all those statistics is the sampling distribution. In practice, we use simulation to approximate it.

✦ KEY TAKEAWAY

Think of it like a vending machine that dispenses sample means instead of snacks. Each time you press the button (take a new random sample), you get a slightly different number. If you pressed that button thousands of times and plotted every number on a histogram, that histogram is the sampling distribution. It tells you which sample means are common (near the center) and which are rare (in the tails).

SECTION 3

Visual Explanation — From Population to Sampling Distribution

The diagram below walks you through the simulation process step by step. On the left you see the original population, in the middle you see individual samples being drawn, and on the right you see the sampling distribution that emerges as those sample means pile up.

The diagram shows a skewed population (left), three representative samples drawn from it (center), and the sampling distribution of the sample mean (right) that forms after 1,000 repetitions. Notice the bell shape even though the population is skewed.

The most striking feature is how the shape transforms. The original population was clearly skewed to the right, but after drawing 1,000 samples of size n = 30 and plotting their means, the resulting histogram looks approximately normal (bell-shaped). This is not a coincidence — it is one of the most important results in statistics, and we'll explore the math behind it in the next section.

SECTION 4

Mathematical Framework

While simulation gives us a hands-on feel for sampling distributions, there are formulas that describe the key properties without needing a computer. These formulas connect the population parameters to the behavior of the sample mean.

MEAN OF THE SAMPLING DISTRIBUTION

μ_x̄ = μ

The mean of the sampling distribution of x̄ equals the population mean μ. This tells us the sampling distribution is centered on the true population parameter — the sample mean is an unbiased estimator.

STANDARD ERROR OF THE MEAN

σ_x̄ = σ / √n

σ = population standard deviation, n = sample size. The standard error (σ_x̄) measures how much sample means typically vary from μ. Larger samples produce a smaller standard error, meaning the sample means cluster more tightly around μ.

CENTRAL LIMIT THEOREM (INFORMAL)

For large n (typically n ≥ 30), the distribution of x̄ is approximately Normal: x̄ ~ N(μ, σ/√n)

The Central Limit Theorem (CLT) says that regardless of the population's shape, the sampling distribution of the sample mean approaches a normal distribution as the sample size increases. This is why our skewed population in the diagram still produced a bell-shaped sampling distribution.

💡 Why does √n matter?

The square root in σ/√n explains a pattern you'll see in practice: doubling your sample size does not cut the spread in half — it only reduces it by a factor of √2 ≈ 1.41. To cut the standard error in half, you need to quadruple the sample size. This has real-world budget implications: polling 4,000 people instead of 1,000 only halves your margin of error.

SECTION 5

How Sample Size Changes the Sampling Distribution

One of the most powerful things a simulation can reveal is the effect of sample size on the shape and spread of the sampling distribution. The diagram below compares three sampling distributions — all from the same population, but with sample sizes of n = 5, n = 30, and n = 100.

Three sampling distributions of x̄, all centered at μ = 50. The amber curve (n = 5) is wide and flat. The pink curve (n = 30) is noticeably tighter. The cyan curve (n = 100) is the narrowest, showing that larger samples give more precise estimates.

Standard error decreases as sample size grows

Sample Size (n)	Standard Error σ/√n	Practical Meaning
5	15 / √5 = 6.71	Sample means vary a lot; any single sample could be far from μ.
30	15 / √30 = 2.74	Much less variability; most sample means fall within about 5.5 points of μ.
100	15 / √100 = 1.50	Very tight cluster; the sample mean is a reliable estimate of μ.

The key insight from both the diagram and the table is that increasing the sample size makes your estimate more precise, but with diminishing returns. Going from n = 5 to n = 30 cuts the standard error nearly in half (from 6.71 to 2.74), but going from n = 30 to n = 100 only reduces it by about another point. This trade-off between effort and precision is a constant theme in real-world statistics.

SECTION 6

Worked Example — Simulating a Sampling Distribution

Let's walk through a complete simulation scenario. A school nurse knows that the resting heart rate for teenagers has a population mean of μ = 72 beats per minute (bpm) with a standard deviation of σ = 8 bpm. She randomly selects 25 students and records their mean heart rate. She repeats this process 500 times using a computer. What can she expect the sampling distribution to look like?

Estimating the Sampling Distribution of Mean Heart Rate

Step 1 — Identify Given ValuesPopulation mean: μ = 72 bpm. Population standard deviation: σ = 8 bpm. Sample size: n = 25. Number of simulated samples: 500.

μ = 72, σ = 8, n = 25

Step 2 — Calculate the Standard ErrorUse the formula σ_x̄ = σ / √n. Substitute: σ_x̄ = 8 / √25 = 8 / 5 = 1.6 bpm. This means the typical distance between any sample mean and the true population mean is about 1.6 bpm.

Standard Error = 1.6 bpm

Step 3 — Determine the ShapeSince n = 25 is reasonably large (close to the common guideline of n ≥ 30), and heart rates are roughly normally distributed to begin with, the Central Limit Theorem tells us the sampling distribution will be approximately normal. When the population itself is close to normal, even smaller sample sizes produce a normal-looking sampling distribution.

Shape ≈ Normal

Step 4 — Describe the Sampling DistributionPutting it all together, the sampling distribution of x̄ is approximately Normal with center μ_x̄ = 72 bpm and standard error σ_x̄ = 1.6 bpm. We write this as x̄ ~ N(72, 1.6).

x̄ ~ N(72, 1.6)

Step 5 — Interpret in ContextAfter running 500 simulations, the nurse should expect her histogram of sample means to be bell-shaped, centered at 72 bpm. About 68% of her 500 sample means should fall between 72 − 1.6 = 70.4 and 72 + 1.6 = 73.6 bpm. About 95% should fall between 72 − 3.2 = 68.8 and 72 + 3.2 = 75.2 bpm.

68% of x̄ values between 70.4 and 73.6 bpm; 95% between 68.8 and 75.2 bpm

SECTION 7

Strengths & Limitations of Simulation

Simulation is a powerful and accessible tool, but like all tools it comes with trade-offs. Understanding when simulation shines — and when it falls short — will help you decide how to approach a problem.

Comparing the advantages and disadvantages of using simulation

Strengths	Limitations
No advanced math needed — just repeatedly sample and plot.	Results vary each time you run the simulation (sampling variability applies to the simulation itself).
Works for any statistic (mean, median, proportion, standard deviation, etc.).	Requires a computer or calculator for a realistic number of repetitions (500+).
Makes abstract concepts visual and concrete — you can literally see the distribution form.	If the simulation model is flawed (e.g., your population model is wrong), results will be misleading.
Doesn't require the population to be normal or any specific shape.	A simulation gives an approximation; mathematical formulas (when available) give exact answers.

✦ KEY TAKEAWAY

Simulation is like using a flight simulator before flying a real plane — it gives you a realistic feel for how things work without the cost or risk of the real thing. It won't replace the physics of flight (the mathematical theory), but it makes the theory tangible. In statistics, running 1,000 simulated samples is the quickest way to build intuition about what a sampling distribution actually looks like.

SECTION 8

Connection to Confidence Intervals & Hypothesis Testing

Everything you've learned about sampling distributions is the launching pad for the two big topics in inferential statistics: confidence intervals and hypothesis testing. Both of these tools rely on knowing how sample statistics behave — which is exactly what the sampling distribution tells you.

How this lesson connects to future topics

Concept	What You've Learned (This Lesson)	What Comes Next
Sampling Distribution	The distribution of a statistic over many repeated samples.	Used as the basis for calculating probabilities in inference.
Standard Error	σ/√n measures the spread of sample means.	Becomes the building block of the margin of error in a confidence interval.
Central Limit Theorem	Guarantees approximate normality for large n.	Justifies using z-scores and t-scores to make probability statements.
Simulation	Estimate a sampling distribution by drawing many random samples.	Extends to bootstrap methods and randomization tests in advanced courses.

When you eventually learn to build a 95% confidence interval, you'll use the fact that approximately 95% of sample means fall within 2 standard errors of μ. That rule comes directly from the normal shape of the sampling distribution guaranteed by the Central Limit Theorem. So the simulation work you do now is not just a classroom exercise — it's the conceptual engine behind real statistical inference.

SECTION 9

Practice Problems

PROBLEM 1 — CONCEPTUAL

Explain in your own words why the sampling distribution of the sample mean is different from the distribution of the population itself. Why isn't the sampling distribution just a copy of the population?

PROBLEM 2 — BASIC CALCULATION

A population has a mean of μ = 100 and a standard deviation of σ = 20. If you take random samples of size n = 64, what is the standard error of the sampling distribution of x̄?

PROBLEM 3 — INTERMEDIATE

A researcher simulates 1,000 random samples of size n = 36 from a population with μ = 50 and σ = 18. She then plots a histogram of the 1,000 sample means. Describe the expected shape, center, and spread of this histogram. If she increased the sample size to n = 144 (but still simulated 1,000 samples), how would the histogram change?

PROBLEM 4 — APPLIED

A quality control engineer at a battery factory knows the population mean battery life is μ = 500 hours with σ = 40 hours. She randomly tests 50 batteries each day and calculates the daily sample mean. Over the course of 200 working days, she plots the 200 sample means. She notices that one day's mean was x̄ = 488 hours. Should she be concerned that something went wrong in the factory that day? Use the properties of the sampling distribution to support your reasoning.

PROBLEM 5 — CRITICAL THINKING

A student runs a simulation: she draws 500 samples of size n = 10 from a population that is heavily right-skewed and plots the sample means. She then draws 500 samples of size n = 50 from the same population and plots those sample means. She claims: 'Both histograms will be perfectly normal because of the Central Limit Theorem.' Evaluate her claim. What would you actually expect to see in each histogram, and why?

SUMMARY

Lesson Summary

A sampling distribution is the distribution of a statistic (such as x̄) computed from many repeated random samples of the same size drawn from a population. Using simulation, you can approximate this distribution by generating hundreds or thousands of samples and plotting the resulting statistics. The sampling distribution of x̄ has three key properties: it is centered at the population mean μ, its spread is measured by the standard error σ/√n, and its shape approaches normal as n increases, thanks to the Central Limit Theorem.

The most important practical takeaway is that increasing the sample size reduces the sampling variability, making your sample statistic a more reliable estimate of the true parameter. This foundational idea connects directly to confidence intervals and hypothesis testing, the core tools of statistical inference that you'll study next.