Model Limitations & Uncertainty — I can analyze model...

SECTION 1

Historical Context & Motivation

Throughout history, humans have built mathematical models to predict everything from planetary orbits to population growth. These models have been extraordinarily useful, but every single one involves trade-offs. A model that captures every detail of reality would be so complex that it defeats the purpose of modeling in the first place. The story of mathematical modeling is really the story of choosing what to include, what to leave out, and then honestly communicating what we don't know.

From ancient astronomers who assumed perfectly circular orbits to modern economists who assume people always behave rationally, the history of modeling is filled with brilliant simplifications that worked well — until they didn't. Understanding when and why a model breaks down is just as important as knowing how to use one.

1609

Kepler Corrects Circular Orbits

For centuries, astronomers assumed planets move in perfect circles. Johannes Kepler showed that elliptical orbits fit the data far better, revealing how a simplifying assumption had limited earlier models.

1798

Malthus and Exponential Growth

Thomas Malthus modeled population growth as exponential. While accurate in the short term, it ignored factors like resource limits and disease — a classic example of a model that fails when extrapolated too far.

1920s

Statistical Confidence Intervals

Statisticians like Ronald Fisher and Jerzy Neyman developed formal methods to quantify uncertainty, giving scientists tools to say not just 'here's our prediction' but 'here's how confident we are.'

2008

Financial Models Fail

The global financial crisis exposed how models that assumed housing prices would always rise collapsed when that assumption broke. Billions of dollars were lost because model limitations were ignored.

2020

Pandemic Modeling Under Uncertainty

COVID-19 models had to be updated constantly as new data emerged. Public communication of uncertainty — showing ranges of possible outcomes — became critical for decision-making.

The central question this lesson addresses is: How do we recognize the boundaries of a model's usefulness, and how do we honestly communicate what a model can and cannot tell us?

SECTION 2

Core Principles & Definitions

Before diving into the mechanics of analyzing model limitations, you need a solid foundation in a few key ideas. Every mathematical model is a simplified representation of a real-world situation. The act of simplifying introduces potential errors, and those errors become more significant under certain conditions. Here are the foundational concepts you'll use throughout this lesson.

Interpolation vs. Extrapolation

Interpolation means using a model to predict values within the range of data you already have. Extrapolation means predicting beyond that range. Extrapolation is riskier because you're assuming patterns continue where you have no evidence.

Simplifying Assumptions

A simplifying assumption is a deliberate choice to ignore certain factors to make the model manageable. For example, assuming air resistance is zero or that growth is constant. Every assumption is a potential source of error.

Domain of Validity

The domain of validity is the range of input values where the model's predictions are trustworthy. Outside this range, the model may produce nonsensical or dangerously wrong results.

Residuals & Error

A residual is the difference between an observed value and the value predicted by the model. Patterns in residuals can reveal systematic problems with the model's structure.

Communicating Uncertainty

Responsible modeling means stating predictions with uncertainty language: confidence intervals, margins of error, or qualitative hedging like 'the model suggests' rather than 'the answer is.'

✦ KEY TAKEAWAY

Think of a mathematical model like a weather app on your phone. It's great at telling you tomorrow's temperature, decent for a 3-day forecast, and nearly useless for predicting the weather 30 days from now. The further you push any model beyond its comfort zone, the less you should trust its predictions. The key skill is knowing where that comfort zone ends.

SECTION 3

Visual Explanation — Interpolation vs. Extrapolation

The diagram below shows a scatter plot of data points with a regression line fit to the data. Notice how the model fits reasonably well within the observed data range (the interpolation zone), but the prediction becomes increasingly unreliable as we extend the line beyond the data (the extrapolation zone). The shaded confidence band widens dramatically once you leave the observed data range.

The green interpolation zone shows where the model is supported by actual data. The red extrapolation zone shows where predictions go beyond the data. Notice how the actual trend (gold dashed line) diverges from the model's straight-line prediction — this is why extrapolation is risky.

In the diagram, the cyan dots represent observed data points. The violet regression line fits well within the data range — the residuals are small and scattered randomly. However, once we cross the red dashed boundary into the extrapolation zone, the model predicts continued linear growth. The gold dashed curve shows what actually happens: the real trend levels off. The model's straight-line assumption fails because the real-world situation involves a factor (like market saturation) that the model ignores.

The purple shaded confidence band illustrates another critical idea: even within the interpolation zone, predictions aren't exact. The band widens as you move away from the center of the data, and it expands dramatically in the extrapolation zone. This widening band is a visual representation of increasing uncertainty.

SECTION 4

Mathematical Framework

Several mathematical tools help you quantify how well a model fits data and how much uncertainty is present in its predictions. These formulas let you move from a gut feeling that 'the model seems okay' to precise numerical statements about model quality.

RESIDUAL

eᵢ = yᵢ − ŷᵢ

where eᵢ is the residual for data point i, yᵢ is the actual observed value, and ŷᵢ is the predicted value from the model. A positive residual means the model underpredicted; a negative residual means it overpredicted.

COEFFICIENT OF DETERMINATION

R² = 1 − (SS_res / SS_tot)

where SS_res = Σ(yᵢ − ŷᵢ)² is the sum of squared residuals and SS_tot = Σ(yᵢ − ȳ)² is the total sum of squares. R² ranges from 0 to 1. A value of 0.85 means the model explains 85% of the variation in the data. Higher is better, but R² alone doesn't guarantee the model is appropriate.

PERCENT ERROR

Percent Error = |actual − predicted| / |actual| × 100%

This measures how far off a single prediction is as a percentage of the actual value. When communicating uncertainty, percent error gives your audience a sense of scale — a 5% error in a weather forecast is very different from a 5% error in a medical dosage.

PREDICTION WITH MARGIN OF ERROR

Prediction = ŷ ± margin of error

Rather than stating a single number, you express the prediction as a range. For example, 'the model predicts sales of 450 ± 30 units' means the true value is likely between 420 and 480. The ± value reflects the model's uncertainty and grows when you extrapolate beyond the data.

⚠️ When R² Can Mislead You

A high R² value does not guarantee that a model is correct. A quadratic model fit to data that follows an exponential pattern might produce R² = 0.95 within the data range but fail catastrophically when you extrapolate. Always look at residual plots — if you see a pattern (like a curve) in the residuals, the model structure is wrong even if R² looks good.

SECTION 5

Types of Model Limitations

Model limitations fall into several categories. Understanding these categories helps you systematically evaluate any model you encounter. The diagram below organizes the most common types of limitations and shows how they relate to one another.

This taxonomy organizes model limitations into three categories: structural limitations (wrong model type or missing variables), domain limitations (extrapolation and context breakdown), and data quality limitations (small or biased samples). All three inform how you should communicate uncertainty.

Let's look at each category in more detail. Structural limitations occur when the mathematical form of the model is wrong. If you fit a straight line to data that follows a curve, no amount of extra data will fix the problem. The model's structure is fundamentally mismatched with reality. A residual plot will reveal this: instead of a random scatter, you'll see a systematic pattern (like a U-shape).

Domain limitations arise when you use a model outside the conditions where it was developed. This includes extrapolation (going beyond the data range) and context changes (the underlying situation shifts). A population growth model built during peacetime may fail completely during a famine, even within the same time range.

Data quality limitations come from problems with the input data itself. If you built a regression model from only five data points, the model is less trustworthy than one built from fifty. Similarly, if your data is biased — for example, surveying only students at one school to predict trends across all schools — the model may generalize poorly.

SECTION 6

Worked Example

A coffee shop tracks its weekly iced coffee sales over 10 weeks in spring and summer, along with the average weekly temperature. They fit a linear model and want to use it to plan inventory. Let's walk through analyzing the model's limitations and communicating uncertainty.

Iced Coffee Sales Data with Linear Model Predictions

Week	Avg Temp (°F)	Iced Coffees Sold	Model Prediction	Residual
1	55	80	78	+2
2	58	95	90	+5
3	62	105	106	−1
4	67	125	126	−1
5	72	148	146	+2
6	75	160	158	+2
7	80	172	178	−6
8	84	190	194	−4
9	88	205	210	−5
10	90	215	218	−3

Analyzing Model Limitations for Iced Coffee Sales

Step 1 — Identify the Model and Its AssumptionsThe linear model is ŷ = 4.0x − 142, where x is the average temperature in °F. This assumes a constant rate of increase: for every 1°F rise in temperature, the model predicts 4.0 more iced coffees sold. The simplifying assumptions include: (1) the relationship is linear, (2) temperature is the only factor that matters, and (3) the trend continues indefinitely.

Model: ŷ = 4.0x − 142, with three key simplifying assumptions identified.

Step 2 — Calculate R² and Assess FitComputing SS_res: sum up all (yᵢ − ŷᵢ)² values. The squared residuals are 4, 25, 1, 1, 4, 4, 36, 16, 25, 9. So SS_res = 125. Computing SS_tot using ȳ ≈ 149.5 gives SS_tot ≈ 18,832. Therefore R² = 1 − (125 / 18,832) ≈ 0.993. This is a very high R², meaning the model explains 99.3% of the variation in the data.

R² ≈ 0.993 — excellent fit within the data range

Step 3 — Examine Residuals for PatternsThe residuals are: +2, +5, −1, −1, +2, +2, −6, −4, −5, −3. Notice a pattern: early residuals tend to be positive (model underpredicts), while later residuals are negative (model overpredicts). This suggests the data might curve slightly, with the true relationship leveling off at higher temperatures. A purely random scatter would show no such pattern.

Residual pattern detected: possible curvature that the linear model misses.

Step 4 — Identify Extrapolation RisksWhat if the owner wants to predict sales when the temperature reaches 105°F? Plugging in: ŷ = 4.0(105) − 142 = 278 iced coffees. But our data only goes up to 90°F. Predicting at 105°F is extrapolation. In reality, when it's extremely hot, people may stay home rather than walk to a coffee shop, so sales could actually drop. The model predicts infinite growth, which is physically impossible.

Predicting at 105°F is unreliable extrapolation — 15°F beyond the data range.

Step 5 — Communicate with Uncertainty LanguageInstead of saying 'We'll sell 278 iced coffees when it's 105°F,' a responsible statement would be: 'Within the observed range of 55°F to 90°F, the model predicts sales well (R² ≈ 0.99). For temperatures between 55°F and 90°F, we expect sales of 4.0 × (temp) − 142, give or take about 5 coffees. We cannot reliably predict sales beyond 90°F because we have no data in that range, and the linear trend may not continue.'

Final statement includes domain of validity, margin of error (±5), and explicit limits on extrapolation.

SECTION 7

Strengths & Limitations of Common Model Types

Different model types have different strengths and weaknesses. The table below compares three common models you encounter in Math 3 — linear, quadratic, and exponential — in terms of when they work well, where they break down, and what simplifying assumptions they make.

Comparison of Common Model Types and Their Limitations

Feature	Linear Model	Quadratic Model	Exponential Model
Form	y = mx + b	y = ax² + bx + c	y = a × bˣ
Key Assumption	Constant rate of change	Rate of change is itself changing linearly	Constant percent rate of change
Works Well For	Short-range trends with steady change	Projectile motion, area optimization	Early population growth, compound interest
Breaks Down When	Growth accelerates or decelerates	Behavior doesn't reverse direction (no max/min)	Growth hits limits (resources, space)
Extrapolation Danger	Predicts negative values where impossible	Predicts infinite growth in one direction	Predicts unrealistically large numbers very quickly
Typical R² Trap	Looks good over small intervals even if curve exists	Overfits noise with three parameters	High R² during growth phase masks upcoming plateau

✦ KEY TAKEAWAY

Choosing a model is like choosing a map. A road map is perfect for a driving trip but useless for hiking a trail. A trail map is perfect for hiking but useless for flying. No single map works for every journey, and no single model works for every prediction. The best modelers aren't the ones who pick the fanciest model — they're the ones who understand which model fits the situation and where that model stops being useful.

SECTION 8

Connection to Advanced Theory

The skills you develop in analyzing model limitations are foundational for advanced coursework in statistics, data science, and engineering. In these fields, uncertainty isn't something you try to eliminate — it's something you learn to measure, communicate, and plan around. The table below shows how the concepts you're learning now scale up into more sophisticated tools.

How Math 3 Concepts Evolve in Advanced Courses

Math 3 Concept	Advanced Version
Residuals (observed − predicted)	Residual analysis with normal probability plots, Durbin-Watson test for autocorrelation
R² (coefficient of determination)	Adjusted R², AIC and BIC for comparing models of different complexity
Margin of error (± some value)	Formal confidence intervals and prediction intervals using t-distributions
Extrapolation warning	Leverage and influence diagnostics that quantify how far a prediction is from the data center
Choosing between model types	Model selection via cross-validation, where you test the model on data it hasn't seen

In AP Statistics, you'll formalize many of the ideas from this lesson. You'll compute exact confidence intervals and learn hypothesis testing to determine whether a model's fit is statistically significant. In college-level data science, you'll encounter machine learning models that can fit almost any pattern — but the concepts of overfitting, extrapolation risk, and communicating uncertainty remain just as important as they are here.

🎯 A Skill for Life

Whether you pursue math, science, business, or the humanities, the ability to critically evaluate claims based on models is one of the most useful skills you can develop. Every time you see a headline like 'Study predicts X,' you should ask: What assumptions were made? How far beyond the data does this prediction go? What's the margin of error? These are the questions that separate informed decision-makers from everyone else.

SECTION 9

Practice Problems

PROBLEM 1 — CONCEPTUAL

A student creates a linear model relating hours of study to exam score using data from 5 to 20 hours of study. She then predicts that a student who studies 60 hours will score 142 out of 100. Explain why this prediction is problematic, using at least two specific concepts from this lesson.

PROBLEM 2 — BASIC CALCULATION

A model predicts that a car's value after t years is V(t) = 25,000 × (0.85)ᵗ. The actual value at year 3 is $14,800. Calculate the residual and the percent error for this prediction.

PROBLEM 3 — INTERMEDIATE

A biologist models a bacteria population with P(t) = 200 × (1.4)ᵗ, where t is time in hours. The model was built using data from t = 0 to t = 8. At t = 15, the model predicts P(15) = 200 × (1.4)¹⁵ ≈ 55,434 bacteria, but the actual count is only 12,000. (a) Identify two specific model limitations that could explain this discrepancy. (b) Write a statement communicating the model's predictions with appropriate uncertainty language.

PROBLEM 4 — APPLIED

A city's parks department fits a quadratic model to predict daily park visitors based on high temperature: V(T) = −2T² + 200T − 4,500, valid for temperatures between 60°F and 95°F. The model has R² = 0.87. (a) According to the model, what temperature maximizes visitors? (b) The parks director wants to use this model to plan staffing for a forecasted 102°F day. Calculate the model's prediction and explain why the director should be cautious. (c) Write a memo to the director that communicates the prediction with appropriate uncertainty.

PROBLEM 5 — CRITICAL THINKING

Two students each build a model to predict the number of streaming subscribers for a new service using 12 months of data. Student A uses a linear model and gets R² = 0.94. Student B uses an exponential model and gets R² = 0.97. Student A's residuals show a random scatter; Student B's residuals show a clear curved pattern. Which model would you trust more for predictions within the data range, and which would you trust more for predictions 2 years in the future? Justify your answer using concepts of model fit, residual analysis, and extrapolation.

SUMMARY

Lesson Summary

Every mathematical model is a simplified version of reality, and understanding its limitations is essential to using it responsibly. The three major categories of limitations are structural limitations (wrong model type or missing variables), domain limitations (extrapolation beyond the data range or context changes), and data quality limitations (small or biased samples). Interpolation — predicting within the data range — is generally reliable, while extrapolation — predicting beyond it — carries increasing risk the further you go.

Key tools for evaluating model quality include residual analysis (looking for patterns that reveal structural problems), the coefficient of determination R² (measuring how much variation the model explains), and percent error (quantifying prediction accuracy). When communicating results, always state the model's domain of validity, report predictions with a margin of error, list the simplifying assumptions, and use hedging language like 'the model suggests' rather than making absolute claims.