Comparing Models — I can compare two models for the same...

SECTION 1

Historical Context & Motivation

Throughout history, scientists and mathematicians have faced a recurring challenge: when data from the real world can be described by more than one equation, how do you decide which equation to trust? The practice of model comparison emerged from this very question. Early astronomers debated whether planetary orbits were circular or elliptical; economists argued over whether markets grew linearly or exponentially; and biologists questioned whether populations leveled off or kept climbing. In every case, the answer came down to comparing candidate models against the data and deciding which one fit better while still making sense in the real world.

1805

Legendre's Method of Least Squares

Adrien-Marie Legendre published the method of least squares, providing a systematic way to measure how well a curve fits a set of data points by minimizing squared differences.

1885

Galton & Regression

Francis Galton introduced regression analysis, allowing researchers to model relationships between variables and compare the predictive power of different models.

1900s

R² and Correlation Coefficients

Statisticians formalized the coefficient of determination (R²) as a standard metric for quantifying how much variation in data a model explains.

1970s

Akaike Information Criterion

Hirotsugu Akaike developed a criterion that balances model fit against complexity, warning that a more complicated model isn't always the better one.

Today

Technology-Driven Modeling

Graphing calculators and software like Desmos and GeoGebra let students fit multiple models to the same data set in seconds, making the skill of comparison more important than ever.

The central question this lesson addresses is straightforward but powerful: when you have two (or more) mathematical models that each attempt to describe the same situation, how do you determine which model is better, and how do you justify your choice? The answer requires both numerical analysis (fit) and human judgment (reasonableness).

SECTION 2

Core Principles of Model Comparison

Comparing models isn't just about picking the curve that looks closest to the data. A rigorous comparison rests on several foundational ideas that balance mathematical precision with practical thinking. Before diving into calculations, it helps to understand the principles that guide every model comparison.

Goodness of Fit

A model's goodness of fit measures how closely its predicted values match the actual data. Common metrics include R² and the sum of squared residuals. The closer the model's curve passes through the data points, the better the fit.

Residual Analysis

A residual is the difference between an observed value and the value predicted by the model (residual = actual − predicted). Examining the pattern and size of residuals reveals whether a model systematically over- or under-predicts.

Reasonableness

A model must make sense in context. Even if a polynomial fits the data perfectly, it might predict negative populations or temperatures above 1,000 °F. Reasonableness means the model's behavior outside the data range is realistic.

Parsimony (Simplicity)

When two models fit equally well, the simpler one is usually preferred. This principle of parsimony prevents overfitting—where a model memorizes noise instead of capturing the true trend.

Domain Appropriateness

Different model types carry different assumptions. An exponential model implies unlimited growth, while a logistic model implies a ceiling. Choosing a model whose underlying assumptions match the real situation is called domain appropriateness.

✦ KEY TAKEAWAY

Think of model comparison like choosing shoes for a hike. A fancy dress shoe might technically fit your foot (good numerical fit), but it would be a terrible choice on a rocky trail (poor reasonableness). The best model is the one that fits the data and makes sense for where you're going—just like hiking boots fit your foot and match the terrain.

SECTION 3

Visualizing Two Models on the Same Data

The most intuitive way to compare models is to plot them on the same set of axes alongside the data. The diagram below shows a set of data points representing the population of a small town over ten years. Two candidate models—a linear model and a quadratic model—are overlaid so you can visually assess which one tracks the data more closely.

The amber dots represent actual population data. The dashed cyan line is the linear model, while the solid pink curve is the quadratic model. Notice how the quadratic curve passes closer to the data points in the later years, suggesting the growth rate is accelerating rather than staying constant.

Looking at the diagram, you can see that both models capture the general upward trend, but the quadratic curve hugs the data points more tightly, especially in years 7 through 10 where the population growth appears to accelerate. The linear model consistently under-predicts in the later years, which suggests that the relationship between time and population is not perfectly linear. Visual inspection is a powerful first step, but we need numerical tools to make a definitive comparison.

SECTION 4

Mathematical Framework for Comparing Models

Visual inspection gives you intuition, but numbers give you evidence. The two primary tools for quantifying model fit are residuals and the coefficient of determination (R²). Together, they let you make an objective, defensible argument about which model is superior.

RESIDUAL

eᵢ = yᵢ − ŷᵢ

Where eᵢ is the residual for data point i, yᵢ is the actual (observed) value, and ŷᵢ is the value predicted by the model. Positive residuals mean the model under-predicted; negative residuals mean it over-predicted.

SUM OF SQUARED RESIDUALS (SSR)

SSR = Σ(yᵢ − ŷᵢ)²

Squaring each residual ensures that positive and negative errors don't cancel out. A smaller SSR indicates a better fit. When comparing two models, the one with the lower SSR fits the data more closely.

COEFFICIENT OF DETERMINATION

R² = 1 − (SSR / SST)

Here SST is the total sum of squares, SST = Σ(yᵢ − ȳ)², where ȳ is the mean of all observed values. R² ranges from 0 to 1; a value closer to 1 means the model explains a larger proportion of the variability in the data.

An R² of 0.95, for example, means the model accounts for 95% of the variation in the data. If one model has R² = 0.92 and another has R² = 0.98 for the same data set, the second model fits better—at least numerically. However, R² alone is not enough. You must also examine whether the residuals show a pattern. If the residual plot reveals a clear curve, the model is systematically missing something, regardless of how high R² might be.

⚠️ Watch Out: Overfitting

A higher-degree polynomial will almost always produce a higher R² because it has more flexibility to bend toward every data point. But a 9th-degree polynomial that passes through all 10 points is memorizing the data, not modeling the trend. This is called overfitting. Always ask: does the added complexity genuinely reflect the real-world behavior, or is the model just chasing noise?

SECTION 5

Residual Plots — The Detective's Tool

A residual plot graphs the residuals (vertical axis) against the independent variable or the predicted values (horizontal axis). The pattern you see in this plot tells you whether your model is appropriate. If the residuals scatter randomly above and below zero with no discernible pattern, the model is a good structural fit. If the residuals form a curve or a funnel, the model is missing something important.

Left: the linear model's residual plot shows a curved pattern (cyan dots arcing downward), indicating systematic error—the linear model is structurally inadequate. Right: the quadratic model's residual plot shows random scatter (pink dots hovering near zero), indicating that it captures the true trend and the remaining errors are random noise.

The key insight from residual analysis is this: R² can be deceivingly high even when a model is structurally wrong. A linear model with R² = 0.94 might seem good, but if its residual plot shows a parabolic curve, a quadratic model will almost certainly be more appropriate. Always check the residual plot before declaring a winner.

Random scatter around zero → the model type is appropriate.
U-shaped or curved pattern → try a higher-degree polynomial or a different function family.
Fan or funnel shape → the variability changes with the data, suggesting a transformation (like logarithmic) may help.
One or two extreme outliers → investigate those data points; they may be errors or special cases.

SECTION 6

Worked Example: Choosing Between Linear and Exponential Models

A scientist records the number of bacteria (in thousands) in a petri dish every hour. She fits two models to the data: a linear model ŷ = 3.2x + 5 (R² = 0.91) and an exponential model ŷ = 5 × 1.45ˣ (R² = 0.99). The data for hours 0 through 6 are: 5, 7.5, 10.5, 15, 22, 31, 46. Which model should she use?

Comparing Linear vs. Exponential Models for Bacteria Growth

Step 1 — Compute Residuals for Both ModelsFor each data point, calculate eᵢ = yᵢ − ŷᵢ using both models. For example, at x = 3 (hour 3), the actual value is 15. The linear model predicts 3.2(3) + 5 = 14.6, giving a residual of 15 − 14.6 = 0.4. The exponential model predicts 5 × 1.45³ ≈ 15.25, giving a residual of 15 − 15.25 = −0.25. Repeating for all points, the linear residuals grow progressively larger in later hours, while the exponential residuals stay small.

Linear residuals: −0.2, −0.9, −1.1, 0.4, 3.0, 6.0, 10.6 | Exponential residuals: 0, −0.25, 0.02, −0.25, 0.37, −0.27, 0.38

Step 2 — Compute SSR for Each ModelSquare each residual and sum them. For the linear model: (−0.2)² + (−0.9)² + (−1.1)² + (0.4)² + (3.0)² + (6.0)² + (10.6)² = 0.04 + 0.81 + 1.21 + 0.16 + 9.0 + 36.0 + 112.36 = 159.58. For the exponential model: (0)² + (0.25)² + (0.02)² + (0.25)² + (0.37)² + (0.27)² + (0.38)² ≈ 0 + 0.0625 + 0.0004 + 0.0625 + 0.1369 + 0.0729 + 0.1444 = 0.48.

SSR (linear) ≈ 159.58 | SSR (exponential) ≈ 0.48

Step 3 — Compare R² ValuesThe linear model has R² = 0.91, meaning it explains 91% of the variation. The exponential model has R² = 0.99, explaining 99%. The exponential model has a dramatically lower SSR and a higher R², indicating a much better numerical fit.

R² (linear) = 0.91 vs. R² (exponential) = 0.99 → exponential fits better

Step 4 — Check Residual PatternsThe linear residuals show a clear trend: they start near zero, then grow dramatically in the positive direction. This curved pattern means the linear model systematically under-predicts at higher values. The exponential residuals scatter randomly around zero with no pattern, indicating the exponential model captures the growth structure well.

Linear residuals → patterned (bad) | Exponential residuals → random (good)

Step 5 — Assess ReasonablenessBacteria reproduce by cell division, which is fundamentally a multiplicative (exponential) process. The linear model implies the same number of bacteria are added each hour, which contradicts the biology. The exponential model reflects the doubling nature of bacterial reproduction and aligns with scientific knowledge. Over a short time frame before resources run out, exponential growth is the standard biological model.

Conclusion: The exponential model is superior. It has a higher R² (0.99 vs. 0.91), a lower SSR (0.48 vs. 159.58), random residuals, and aligns with the biological mechanism of cell division.

SECTION 7

Strengths and Limitations of Common Model Types

Not every model type is appropriate for every situation. Understanding the built-in assumptions and behaviors of common model families helps you narrow down candidates before you even look at R² or residuals. The table below summarizes when each model type shines and when it falls short.

Common model types with their ideal uses and limitations

Model Type	Best Used When…	Limitations
Linear (y = mx + b)	Data shows a constant rate of change; the scatter plot forms a roughly straight pattern.	Cannot capture acceleration, deceleration, or leveling off. Poor for long-range predictions when growth is non-constant.
Quadratic (y = ax² + bx + c)	Data has one turning point (a maximum or minimum), such as projectile height or profit optimization.	Predicts values going to ±∞ in both directions, which is unrealistic for many real situations.
Exponential (y = a × bˣ)	Data grows (or decays) by a constant percentage each period; multiplicative processes like population growth or radioactive decay.	Predicts unlimited growth or decay, which eventually breaks down when resources are limited.
Logarithmic (y = a + b ln x)	Data rises quickly at first and then levels off; diminishing returns situations.	Only defined for positive x values. Can under-predict if the leveling off is temporary.
Logistic (y = L / (1 + e⁻ᵏ⁽ˣ⁻ˣ⁰⁾))	Data shows S-shaped growth with a carrying capacity; spread of diseases, adoption of technology.	Requires estimating three parameters (L, k, x₀), which can be difficult with limited data.

✦ KEY TAKEAWAY

Choosing a model is like choosing a vehicle for a road trip. A sports car (linear model) is fast and simple for a smooth highway, but if the road starts curving through mountains (non-linear data), you need something with better handling—a quadratic or exponential model. And if the road eventually levels onto a plateau, a logistic model is the SUV that can handle the full journey. The best model matches the shape of the road, not just the starting stretch.

SECTION 8

Looking Ahead — Advanced Model Comparison Tools

In this course, R², SSR, residual plots, and reasonableness checks are your primary comparison tools. But as you advance in statistics and data science, you will encounter more sophisticated methods that build on these same ideas. The table below shows how the tools you're learning now connect to what comes next.

Current tools and their advanced counterparts

What You Use Now	Advanced Version	What It Adds
R² (coefficient of determination)	Adjusted R²	Penalizes adding unnecessary variables, preventing overfitting when comparing models with different numbers of parameters.
Visual residual plots	Formal residual tests (Durbin-Watson, Breusch-Pagan)	Provides statistical p-values to determine if patterns in residuals are significant rather than just visual impressions.
Reasonableness check	Cross-validation	Tests the model on data it hasn't seen yet, revealing whether it generalizes or just memorizes the training data.
SSR comparison	AIC / BIC (Information Criteria)	Balances fit quality against model complexity using information theory—lower is better.

The core philosophy never changes: a good model should fit the data, be structurally appropriate, and remain useful for prediction. The advanced tools simply automate and quantify what you are already doing by hand—checking fit, checking patterns, and checking reasonableness. Mastering the fundamentals now will make these advanced techniques feel like natural extensions rather than completely new concepts.

SECTION 9

Practice Problems

PROBLEM 1 — CONCEPTUAL

A student fits a linear model to a data set and gets R² = 0.96. She claims this proves the linear model is the best choice. Explain why she might be wrong, and describe one additional step she should take before making her conclusion.

PROBLEM 2 — BASIC CALCULATION

A data set has actual values y = {4, 7, 12, 19}. Model A predicts ŷ = {5, 8, 11, 18} and Model B predicts ŷ = {3, 7, 13, 19}. Calculate the SSR for each model and state which one fits the data more closely.

PROBLEM 3 — INTERMEDIATE

A car's value (in thousands of dollars) over 5 years is: {25, 20, 16.5, 13, 11}. A linear model gives ŷ = −3.5x + 27 with R² = 0.97. An exponential model gives ŷ = 27 × 0.82ˣ with R² = 0.995. Calculate the residuals for both models at year 3, compare the R² values, and determine which model is more appropriate. Justify your choice using both fit and reasonableness.

PROBLEM 4 — APPLIED

A city's health department tracks flu cases over 12 weeks. An exponential model ŷ = 10 × 1.6ˣ fits well for weeks 1–6 (R² = 0.98), but by week 12 it predicts 11,500 cases when the actual number is 3,200. A logistic model ŷ = 4000 / (1 + 399e⁻⁰·⁸ˣ) fits the full 12-week data with R² = 0.97. Which model should the health department use for planning, and why? Discuss both fit and reasonableness in your answer.

PROBLEM 5 — CRITICAL THINKING

You have 8 data points and fit three models: a linear model (R² = 0.88), a quadratic model (R² = 0.94), and a 7th-degree polynomial (R² = 1.00). A classmate argues that the 7th-degree polynomial is the best model because its R² is perfect. Write a well-reasoned paragraph explaining why this argument is flawed. Reference at least three specific concepts from this lesson.

SUMMARY

Lesson Summary

Comparing mathematical models requires both numerical evidence and real-world judgment. Start by calculating residuals (eᵢ = yᵢ − ŷᵢ) for each candidate model and computing the sum of squared residuals (SSR) to quantify overall error. Use the coefficient of determination (R²) to measure how much variation each model explains—values closer to 1 indicate a stronger fit. But never stop at R² alone: examine the residual plot to check whether errors are randomly scattered (good) or form a pattern (bad, suggesting the wrong model type).

Beyond fit statistics, always assess reasonableness: does the model's behavior make sense for the real-world context, especially when extrapolating beyond the data? Apply the principle of parsimony by preferring simpler models when they fit comparably well, and guard against overfitting—where a complex model memorizes data noise rather than capturing the true trend. The strongest justification for choosing a model combines a high R², randomly scattered residuals, structural simplicity, and alignment with the real-world situation.