Understanding Probability — I can interpret probability as...

SECTION 1

Historical Context & Motivation

Humans have been gambling and making bets for thousands of years, but the formal mathematics of probability only began to take shape in the 1600s. Before that era, people relied on superstition, intuition, or gut feelings to assess how likely something was to happen. The question that launched probability theory was surprisingly simple: how should two players fairly split the pot in an unfinished game of chance? That question—known as the Problem of Points—sparked a famous exchange of letters between two French mathematicians, and the results changed how we understand uncertainty forever.

1654

Pascal & Fermat Correspondence

Blaise Pascal and Pierre de Fermat exchanged letters solving the Problem of Points, laying the groundwork for modern probability theory by assigning numerical values to uncertain outcomes.

1713

Bernoulli's Ars Conjectandi

Jacob Bernoulli published his landmark work proving the Law of Large Numbers, which showed that relative frequency converges to the true probability as the number of trials increases.

1814

Laplace's Classical Definition

Pierre-Simon Laplace formally defined probability as the ratio of favorable outcomes to total equally likely outcomes, giving us the formula P(A) = favorable / total.

1933

Kolmogorov's Axioms

Andrey Kolmogorov published his axioms of probability, establishing that every probability is a number between 0 and 1, and providing the rigorous mathematical foundation still used today.

The central question these mathematicians answered is one you face every day: How do I put a number on uncertainty? Whether you're checking the weather forecast, deciding whether to study for a quiz, or wondering about the odds in a card game, you're thinking about probability. This lesson will show you how to assign a precise numerical value between 0 and 1 to any event, and how repeating an experiment many times reveals what that number really means.

SECTION 2

Core Principles & Definitions

Before we start calculating, we need a shared vocabulary. Probability is built on a few key ideas that connect the abstract math to real-world experiments. An experiment is any process whose result is uncertain—flipping a coin, rolling a die, or drawing a card from a shuffled deck. Each possible result is called an outcome, and a collection of one or more outcomes that we care about is called an event.

Probability Scale: 0 to 1

Every probability is a number from 0 (impossible) to 1 (certain). A probability of 0.5 means the event is equally likely to happen or not happen.

Theoretical Probability

When all outcomes are equally likely, probability equals the number of favorable outcomes divided by the total number of outcomes in the sample space.

Experimental (Empirical) Probability

Calculated by actually performing an experiment: relative frequency = (number of times the event occurred) ÷ (total number of trials). This value changes from experiment to experiment.

Long-Run Relative Frequency

As the number of trials grows very large, the experimental probability settles down and approaches the true theoretical probability. This is the Law of Large Numbers in action.

Complement Rule

The probability that an event does not happen equals 1 − P(event). Since the total probability of all possible outcomes is always 1, an event and its complement together cover everything.

✦ KEY TAKEAWAY

Think of probability like a volume knob on a speaker. The knob can't go below 0 (silence—impossible) or above 1 (maximum—certain). Every event has its own volume setting somewhere on that dial. When you flip a fair coin, the knob for "heads" sits right in the middle at 0.5. The further right the knob turns, the more likely the event. If you could somehow run the experiment forever, the fraction of times the event occurs would match exactly where the knob is set—that's long-run relative frequency.

SECTION 3

Visual Explanation — The Probability Scale

The diagram below shows the full probability scale from 0 to 1. Each labeled marker represents a familiar event whose probability you can probably estimate intuitively. Notice how events cluster at different points along the number line, and how the descriptive language shifts from "impossible" on the left to "certain" on the right.

The probability number line runs from 0 (impossible, shown in red) to 1 (certain, shown in cyan). Familiar events are placed at their approximate positions. Notice that a fair coin flip sits exactly at the midpoint, 0.50.

As you read the diagram from left to right, the language shifts from impossible through unlikely, even chance, and likely, all the way to certain. Every probability you will ever calculate lands somewhere on this scale. A probability of exactly 0 means the event cannot occur under the conditions of the experiment, and a probability of exactly 1 means the event must occur. Most interesting events fall strictly between these extremes, and the closer the value is to 1, the more confident you can be that the event will happen on any given trial.

SECTION 4

Mathematical Framework

Now let's formalize the ideas from the previous sections. There are two main formulas you need: one for theoretical probability (when you can count outcomes) and one for experimental probability (when you run actual trials). Both produce a number between 0 and 1.

THEORETICAL PROBABILITY

P(A) = Number of favorable outcomes ÷ Total number of equally likely outcomes

P(A) = the probability of event A occurring. Favorable outcomes are the outcomes that satisfy event A. Total outcomes is the size of the sample space S. This formula requires all outcomes to be equally likely.

EXPERIMENTAL (EMPIRICAL) PROBABILITY

P̂(A) = Number of times A occurred ÷ Total number of trials (n)

P̂(A) (read "P-hat of A") is the relative frequency of event A. Unlike theoretical probability, this value depends on the specific experiment you ran and can vary each time.

LAW OF LARGE NUMBERS

As n → ∞, P̂(A) → P(A)

As the number of trials n grows without bound, the experimental relative frequency P̂(A) converges to the true theoretical probability P(A). This is the formal bridge between theory and experiment.

COMPLEMENT RULE

P(A') = 1 − P(A)

A' ("A complement" or "not A") is the event that A does not occur. Since the probabilities of all possible outcomes must sum to 1, knowing P(A) immediately tells you P(A').

These four formulas are the core toolkit for this unit. The theoretical probability formula works when you can list all equally likely outcomes—think dice, coins, or well-shuffled cards. When the situation is too complex to list outcomes, or when you want to verify a theoretical prediction, you turn to the experimental formula and rely on the Law of Large Numbers to justify that enough trials will reveal the truth.

SECTION 5

Long-Run Relative Frequency in Action

The most powerful idea in this lesson is how experimental results behave as you increase the number of trials. Imagine flipping a fair coin 10 times—you might get 7 heads and 3 tails, giving a relative frequency of 0.70 for heads. That's quite far from the theoretical 0.50. But if you flip the coin 100 times, you'll likely see something closer to 50 heads. By 1,000 flips, the relative frequency will hover very close to 0.50. The diagram below illustrates this convergence.

Two independent simulations of a fair coin are shown. Early in each simulation (small n), the relative frequency bounces wildly. As the number of flips grows into the hundreds and thousands, both lines converge toward the dashed gold line at P = 0.50. This convergence is the Law of Large Numbers in action.

The key takeaway from this graph is that short-run results can be misleading. If you flip a coin only 10 times, getting 8 heads doesn't mean the coin is unfair—it means 10 trials simply aren't enough to reveal the true probability. The long-run relative frequency is what matters, and "long-run" typically means hundreds or thousands of repetitions. This is why weather forecasts say things like "30% chance of rain." Meteorologists are claiming that, over many days with identical atmospheric conditions, rain would occur about 30 out of every 100 times.

⚠️ COMMON MISCONCEPTION

After flipping 5 heads in a row, many people feel that tails is "due." This is called the Gambler's Fallacy. A fair coin has no memory—each flip is independent, and the probability of heads on the next flip is still 0.50, no matter what happened before. The Law of Large Numbers works by diluting past streaks over many future trials, not by producing "compensating" results.

SECTION 6

Worked Example — Dice and Relative Frequency

Let's work through a full example that uses both theoretical and experimental probability and then connects them via long-run relative frequency.

Rolling a Standard Six-Sided Die

Step 1 — Define the Experiment and EventThe experiment is rolling a fair six-sided die once. The sample space is S = {1, 2, 3, 4, 5, 6}. Let event A = "rolling a number less than 3," so A = {1, 2}.

Step 2 — Calculate Theoretical ProbabilityAll six faces are equally likely. There are 2 favorable outcomes (1 and 2) out of 6 total outcomes. Using the formula:

P(A) = 2 ÷ 6 = 1/3 ≈ 0.333

Step 3 — Simulate an Experiment (Small Sample)Suppose you roll the die 12 times and record: 4, 1, 6, 2, 5, 2, 3, 1, 6, 4, 5, 1. The event A occurred for rolls of 1, 2, 2, 1, and 1—that's 5 times out of 12 trials.

P̂(A) = 5 ÷ 12 ≈ 0.417

Step 4 — Compare to TheoryThe experimental probability of 0.417 is higher than the theoretical 0.333. This discrepancy is normal with only 12 trials. The experimental value is within roughly 0.08 of the theoretical value, which is a typical level of fluctuation for a small sample.

Step 5 — Long-Run PredictionIf you rolled the die 10,000 times, the Law of Large Numbers guarantees that the relative frequency of rolling a number less than 3 would converge very close to 1/3. You might see something like 3,321 out of 10,000 trials, giving P̂(A) = 0.3321—extremely close to 0.333.

As n → ∞, P̂(A) → P(A) = 1/3

SECTION 7

Theoretical vs. Experimental Probability

You now know two ways to find a probability: calculate it from equally likely outcomes, or estimate it by running trials. Each approach has strengths and limitations, and understanding when to use which one is an important skill.

Comparison of Theoretical and Experimental Probability

Feature	Theoretical Probability	Experimental Probability
Based on	Counting equally likely outcomes using logic or formulas	Collecting data from actual trials or simulations
Requires	Complete knowledge of the sample space and equally likely outcomes	Ability to perform or simulate the experiment many times
Precision	Exact—gives the true probability directly	Approximate—improves with more trials
Best used when	Outcomes are symmetric, countable, and well-understood (coins, dice, cards)	The situation is too complex to model, or you want to verify a theoretical prediction
Limitation	Cannot be used when outcomes are not equally likely or the sample space is unknown	Small sample sizes can produce misleading results; requires many trials for accuracy

✦ KEY TAKEAWAY

Think of theoretical probability as a recipe and experimental probability as tasting the dish. The recipe tells you exactly what the cake should taste like in theory, but actually baking and tasting it (running trials) confirms whether the recipe works. If you only take one tiny bite (few trials), you might get a misleading sample; taste the whole cake (many trials), and you'll get a reliable verdict. The Law of Large Numbers guarantees that enough bites always converge to the true flavor.

SECTION 8

Connection to Advanced Probability

The ideas in this lesson form the foundation for much more advanced work in statistics and probability. Understanding the 0-to-1 scale and long-run relative frequency prepares you for topics you'll encounter later in high school and in college. Here's a preview of how the concepts you've learned today extend into more complex territory.

From foundational to advanced probability concepts

This Lesson's Concept	Advanced Extension
Probability as a number from 0 to 1	Kolmogorov's axioms formalize this into a rigorous mathematical framework where probability is a function from events to real numbers satisfying three axioms.
P(A) = favorable ÷ total	Conditional probability P(A\|B) = P(A ∩ B) ÷ P(B) extends this idea to situations where you already know something about the outcome.
Long-run relative frequency	Probability distributions (binomial, normal) describe the full pattern of how relative frequencies distribute across all possible outcomes, not just one event.
Law of Large Numbers	The Central Limit Theorem goes further: it tells you that the average of many independent trials follows an approximately normal (bell-curve) distribution, regardless of the original distribution.

You don't need to master these advanced topics right now. The important point is that the simple 0-to-1 scale and the idea that relative frequency stabilizes over many trials are not just introductory ideas—they are the permanent foundation of all probability and statistics. Every insurance actuary, data scientist, medical researcher, and machine learning engineer relies on these same principles.

SECTION 9

Practice Problems

PROBLEM 1 — CONCEPTUAL

A weather app says there is a 0% chance of snow in Miami tomorrow. Another app says there is a 100% chance of the sun rising. Explain what these probability values of 0 and 1 mean, and state whether any probability can ever be less than 0 or greater than 1.

PROBLEM 2 — BASIC CALCULATION

A bag contains 4 red marbles, 3 blue marbles, and 5 green marbles. You draw one marble at random. Find the theoretical probability of drawing a blue marble. Express your answer as a fraction and as a decimal.

PROBLEM 3 — INTERMEDIATE

A basketball player attempts 80 free throws during practice and makes 58 of them. (a) Calculate the experimental probability that she makes a free throw. (b) If the player's true (theoretical) free-throw probability is 0.75, is 80 attempts enough to be confident in the experimental estimate? Explain using the Law of Large Numbers.

PROBLEM 4 — APPLIED

A quality-control inspector tests 500 smartphones from a factory and finds 12 defective units. (a) What is the experimental probability that a randomly selected phone is defective? (b) If the factory produces 20,000 phones per month, use the experimental probability to estimate how many defective phones will be produced. (c) How could the inspector improve the reliability of this estimate?

PROBLEM 5 — CRITICAL THINKING

Marcus rolls a fair die 30 times and gets a "6" on 9 of those rolls. He concludes that the die must be loaded because 9/30 = 0.30, which is much higher than the theoretical probability of 1/6 ≈ 0.167. Evaluate Marcus's reasoning. In your response, address whether 30 trials is sufficient to draw such a conclusion and describe what Marcus should do to test his hypothesis more rigorously.

SUMMARY

Lesson Summary

Probability assigns every event a number on the 0-to-1 scale, where 0 means impossible and 1 means certain. When outcomes are equally likely, you calculate theoretical probability as favorable outcomes divided by total outcomes. When you collect real data, you calculate experimental probability (relative frequency) as the number of times the event occurred divided by the total number of trials. The complement rule tells you that P(not A) = 1 − P(A), since all probabilities must add up to 1.

The most important connection in this lesson is the Law of Large Numbers: as you repeat an experiment more and more times, the long-run relative frequency converges toward the true theoretical probability. Short-run results can be wildly variable, so avoid falling for the Gambler's Fallacy—past outcomes don't influence future independent trials. Whether you're analyzing dice, forecasting weather, or evaluating medical treatments, these foundational principles stay the same.