Comparing Models — I can compare two models for the same situation and justify which is better based on fit and reasonableness. - Grade 11 Math

SECTION 1

Historical Context & Motivation

Throughout history, scientists and mathematicians have faced a recurring challenge: when multiple equations can describe the same phenomenon, how do you decide which one to trust? The practice of model comparison has roots stretching back centuries, arising naturally whenever people tried to predict eclipses, population growth, or economic trends. The core question has always been the same: does this equation actually capture what is happening in the real world, or is there a better one?

1609 • Kepler vs. Ptolemy

Johannes Kepler compared circular-orbit models to elliptical-orbit models for planetary motion. His elliptical model fit observational data far more accurately, replacing centuries of complex epicycle adjustments.

1805 • Legendre's Least Squares

Adrien-Marie Legendre published the method of least squares, giving mathematicians a rigorous, numerical way to measure how well a model fits data by minimizing the sum of squared differences between predicted and observed values.

1885 • Galton's Regression Line

Francis Galton introduced the concept of regression to the mean, leading to linear regression as a standard tool for fitting models to data and comparing their explanatory power.

1973 • Akaike Information Criterion

Hirotsugu Akaike developed the AIC, a formal criterion for comparing models that balances fit against complexity — the idea that a simpler model that fits almost as well may be preferable to a complex one.

Today, model comparison is everywhere — from choosing between a linear and exponential model for COVID case data to deciding whether a quadratic or logarithmic curve best predicts a company's revenue. The central question this lesson addresses is: given the same data set, how do you rigorously compare two models and justify which is better?

SECTION 2

Core Principles of Model Comparison

Before you can compare two models, you need a shared vocabulary and a set of criteria. Model comparison rests on several foundational ideas that guide you from raw data to a justified conclusion. Understanding these principles will prevent you from simply picking whichever equation "looks right" and instead give you tools to defend your choice with evidence.

Residuals

A residual is the difference between an observed data value and the value predicted by a model: residual = observed − predicted. Smaller residuals mean a closer fit.

Coefficient of Determination (R²)

The R² value tells you what fraction of the variation in the data is explained by the model. R² = 1 means a perfect fit; R² = 0 means the model explains nothing.

Residual Patterns

Even if R² is high, you must check whether residuals show a random scatter or a systematic pattern. A curved pattern in the residuals signals that a different type of model may be more appropriate.

Reasonableness

A model must make sense in context. Reasonableness means the model produces realistic predictions — it shouldn't predict negative populations or infinite temperatures.

Parsimony (Simplicity)

When two models fit equally well, prefer the simpler one. Parsimony reduces the risk of overfitting — capturing noise rather than the true underlying trend.

✦

KEY TAKEAWAY

Think of model comparison like choosing between two GPS routes. Both get you to the destination, but you pick the one with the shortest estimated time (best fit) that also avoids roads that are closed (reasonableness). A route that claims zero travel time is a bad model even if it looks great on paper — it doesn't match reality.

SECTION 3

Visualizing Two Models on the Same Data

The most powerful first step in comparing models is to graph them both on the same set of axes alongside the original data. The diagram below shows a scatter plot of data points representing the number of subscribers to a streaming service over its first eight months, with both a linear model and an exponential model overlaid.

The yellow dots are the actual data. The dashed cyan line is the linear model (y = 5x + 1), and the solid pink curve is the exponential model (y = 2 · 1.5^x). Notice how the linear model under-predicts later months while the exponential curve hugs the data more closely throughout.

From the graph alone you can see that the linear model consistently misses the data in the later months — it predicts too few subscribers at month 7 and too many at month 1. The exponential curve tracks the upward acceleration more faithfully. But visual inspection is only the beginning; in the next sections you will learn how to quantify this difference.

SECTION 4

Mathematical Framework for Model Comparison

Visuals help build intuition, but a rigorous comparison requires numbers. The two primary quantitative tools are residual analysis and the coefficient of determination. Both give you concrete evidence you can cite when justifying your model choice.

RESIDUAL

eᵢ = yᵢ − ŷᵢ

where eᵢ is the residual for data point i, yᵢ is the observed value, and ŷᵢ is the predicted value from the model.

SUM OF SQUARED RESIDUALS (SSR)

SSR = Σ(yᵢ − ŷᵢ)²

The SSR totals the squared residuals across all data points. A smaller SSR means the model's predictions are collectively closer to the actual data. Squaring ensures negative and positive errors don't cancel out.

COEFFICIENT OF DETERMINATION

R² = 1 − (SSR / SST)

where SST = Σ(yᵢ − ȳ)² is the total sum of squares (total variation in the data around its mean ȳ). R² ranges from 0 to 1; values closer to 1 indicate a better fit.

⚠

R² Is Not Everything

A high R² does not automatically make a model the best choice. You must also check that the residual plot shows random scatter (no curves or fans) and that the model produces reasonable predictions for values inside and slightly beyond the data range.

SECTION 5

Residual Plots — Reading the Pattern

A residual plot graphs the residuals (vertical axis) against the independent variable or predicted values (horizontal axis). If the model is appropriate, residuals should look like a random cloud of points centered on zero. A clear curve or pattern tells you the model is systematically missing something. The diagram below compares the residual plots for the linear and exponential models from Section 3.

Left: the linear model's residuals form a clear U-shaped curve, indicating it systematically under-predicts at the extremes and over-predicts in the middle. Right: the exponential model's residuals scatter randomly around zero with no obvious pattern — a sign of a good fit.

The curved pattern on the left is the smoking gun. Even though the linear model might have a decent R² value, the residual plot reveals that a linear equation is the wrong type of model for this data. The random scatter on the right confirms that the exponential model captures the underlying trend without systematic error.

✓ Random scatter around zero → the model type is appropriate.
✓ Curved pattern → try a different model type (e.g., switch from linear to quadratic or exponential).
✓ Fanning pattern (residuals grow larger as x increases) → the model's accuracy varies across the data range and may need a transformation.

SECTION 6

Worked Example — Choosing the Better Model

A biology class measured the area (cm²) of a bacterial colony over 6 hours. Two regression models were generated: a linear model y = 3.2x + 0.5 (R² = 0.91) and a quadratic model y = 0.4x² + 0.8x + 1.0 (R² = 0.98). Which model is better, and why?

Hour (x)	Observed Area (y)	Linear ŷ	Quadratic ŷ
0	1.0	0.5	1.0
1	2.1	3.7	2.2
2	3.8	6.9	4.2
3	7.0	10.1	7.0
4	10.5	13.3	10.6
5	15.2	16.5	15.0
6	20.8	19.7	20.2

Bacterial colony area data with predictions from both models

Comparing Linear vs. Quadratic Models

1
Step 1 — Compare R² Values

The linear model has R² = 0.91, meaning it explains 91% of the variation in the data. The quadratic model has R² = 0.98, explaining 98% of the variation. The quadratic model explains more of the data's variability.

Quadratic R² = 0.98 > Linear R² = 0.91
2
Step 2 — Calculate Residuals and SSR

For the linear model, compute each residual eᵢ = yᵢ − ŷᵢ: (0.5, −1.6, −3.1, −3.1, −2.8, −1.3, 1.1). Then SSR = 0.25 + 2.56 + 9.61 + 9.61 + 7.84 + 1.69 + 1.21 = 32.77. For the quadratic model, residuals are: (0, −0.1, −0.4, 0, −0.1, 0.2, 0.6). SSR = 0 + 0.01 + 0.16 + 0 + 0.01 + 0.04 + 0.36 = 0.58.

SSR(linear) = 32.77 vs. SSR(quadratic) = 0.58
3
Step 3 — Examine Residual Patterns

The linear residuals (0.5, −1.6, −3.1, −3.1, −2.8, −1.3, 1.1) start positive, swing strongly negative, then return toward positive — a classic U-shape. The quadratic residuals hover near zero with no systematic curve. This pattern confirms the linear model is the wrong type.

Linear residuals show a curved pattern; quadratic residuals show random scatter.
4
Step 4 — Check Reasonableness

At x = 0, the quadratic gives y = 1.0 (matching the observed starting area), while the linear gives y = 0.5 (below the actual). Predicting ahead to x = 8: the quadratic gives y = 0.4(64) + 0.8(8) + 1.0 = 33.0 cm², which is a plausible growth amount. The linear gives y = 3.2(8) + 0.5 = 26.1, likely an underestimate given the accelerating trend. Both produce positive values, which is physically reasonable for area.

Both models produce reasonable values, but the quadratic's predictions align better with the accelerating growth trend.
5
Step 5 — State Your Justified Conclusion

The quadratic model y = 0.4x² + 0.8x + 1.0 is the better model. It has a higher R² (0.98 vs. 0.91), a dramatically lower SSR (0.58 vs. 32.77), residuals that scatter randomly rather than showing a curved pattern, and predictions that more accurately reflect the accelerating growth of the bacterial colony.

The quadratic model is the better fit based on R², SSR, residual pattern, and reasonableness.

SECTION 7

Comparison Criteria — Strengths & Limitations

No single criterion tells the whole story. The table below summarizes the tools you have for comparing models, along with what each one does well and where it falls short. A strong justification references multiple criteria rather than relying on just one.

Criterion	Strength	Limitation
R² value	Easy to compute and interpret; gives a quick sense of overall fit quality.	Always increases (or stays the same) when you add more terms, so it can reward overly complex models.
SSR	Directly measures total prediction error; lower is always better for the same data set.	Hard to interpret in isolation without context — a SSR of 10 could be great or terrible depending on the data scale.
Residual plot	Reveals whether the model type is appropriate; detects patterns that R² alone cannot.	Subjective — what counts as a "pattern" versus random noise can be debatable with small data sets.
Reasonableness	Ensures the model makes real-world sense; catches models that fit data but produce absurd predictions.	Requires domain knowledge — you need to understand the context to judge what counts as "reasonable."
Parsimony	Guards against overfitting; simpler models tend to generalize better to new data.	Sometimes a more complex model is genuinely needed — simplicity for its own sake can mean poor fit.

Summary of model comparison criteria

✦

KEY TAKEAWAY

Imagine you're a judge at a cooking competition. You wouldn't pick the winner based only on presentation (R²). You'd also taste the food (residuals), check if the ingredients are fresh (reasonableness), and consider whether a simpler recipe achieved the same flavor (parsimony). A thorough model comparison works the same way — use all the criteria together.

SECTION 8

Connection to Advanced Model Selection

The comparison techniques you have learned — R², residual analysis, and reasonableness checks — form the foundation for more sophisticated methods used in statistics, data science, and machine learning. As you progress in mathematics, you'll encounter tools that formalize the trade-off between fit and complexity.

What You Know Now	Where It Leads
R² — measures explained variation	Adjusted R² — penalizes adding unnecessary variables so you can compare models with different numbers of terms fairly
Visual residual plots — look for patterns	Statistical tests (e.g., Durbin-Watson test) that quantify whether residual patterns are statistically significant
Parsimony — prefer simpler models	AIC and BIC — information criteria that mathematically balance fit against the number of parameters
Reasonableness — check predictions	Cross-validation — splitting data into training and test sets to see how well the model predicts unseen values

From Grade 11 techniques to advanced model selection

The key insight that carries forward is this: a model is only as good as its predictions. Whether you're fitting a line in Grade 11 math or training a neural network in college, the fundamental question remains: does this model capture the real pattern in the data, or is it just memorizing noise? The skills you're building now — checking fit, spotting patterns in residuals, and asking whether predictions make sense — are exactly the skills that professional data scientists use every day.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

A student fits a linear model and a quadratic model to the same data set. The linear model has R² = 0.85 and a residual plot with a clear U-shaped curve. The quadratic model has R² = 0.84 and a residual plot with random scatter. Which model would you choose, and why?

Show answer

You should choose the quadratic model despite its slightly lower R². The U-shaped pattern in the linear model's residual plot indicates the linear model is systematically missing the data's curvature — it is the wrong type of model. The quadratic model's random residual scatter means it appropriately captures the data's shape. A small difference in R² (0.85 vs. 0.84) is far less important than a clear pattern in the residuals, which reveals a structural mismatch between model and data.

Question 2: BASIC CALCULATION

Two models predict the temperature (°F) at a campsite over 5 hours. The observed temperatures are {58, 62, 64, 63, 60}. Model A predicts {57, 63, 65, 64, 59} and Model B predicts {60, 60, 60, 60, 60}. Calculate the SSR for each model. Which fits better?

Show answer

For Model A: residuals = (1, −1, −1, −1, 1), so SSR = 1 + 1 + 1 + 1 + 1 = 5. For Model B: residuals = (−2, 2, 4, 3, 0), so SSR = 4 + 4 + 16 + 9 + 0 = 33. Model A has a much lower SSR (5 vs. 33), meaning its predictions are collectively much closer to the actual temperatures. Model A is the better fit. Model B is essentially predicting the mean, which ignores the trend entirely.

Question 3: INTERMEDIATE

A car dealership tracks monthly sales over 12 months. A linear model gives R² = 0.72 with randomly scattered residuals. An exponential model gives R² = 0.89 but predicts that by month 24, sales will reach 50,000 cars per month — the dealership's total capacity is only 200 cars. Which model would you recommend and what criteria support your choice?

Show answer

Despite the exponential model's higher R², its prediction of 50,000 cars per month at month 24 is wildly unreasonable given the dealership's 200-car capacity. This fails the reasonableness criterion. The linear model, while having a lower R² of 0.72, has randomly scattered residuals (indicating the model type is appropriate) and will produce predictions that stay within a plausible range. You would recommend the linear model for the 12-month data range, while noting that a logistic model (which levels off at a maximum) might be even better if the dealership expects growth to plateau.

Question 4: APPLIED

An environmental scientist measures the concentration of a pollutant (ppm) in a lake over 10 weeks of a cleanup effort. She fits two models: Model L (linear): y = −4.5x + 85 with R² = 0.93, and Model E (exponential decay): y = 90 · 0.92ˣ with R² = 0.97. The linear model predicts the pollutant will reach 0 ppm at about week 19. The exponential model predicts 0 ppm is never quite reached but the level drops to 1 ppm by week 54. Which model should she present in her report? Justify using at least three criteria.

Show answer

The exponential decay model is better for three reasons. First, R²: the exponential model explains 97% of the variation versus 93% for the linear model. Second, reasonableness: the linear model predicts the pollutant hits exactly 0 ppm at week 19 and goes negative after that, which is physically impossible — concentrations cannot be negative. The exponential model approaches zero asymptotically, which matches the chemistry of dilution and decomposition processes. Third, residual analysis would likely show the linear model's residuals curving upward at later weeks (since the actual cleanup rate slows as the concentration drops), while the exponential residuals would scatter randomly. The scientist should present the exponential model, noting that real-world concentrations approach but may never perfectly reach zero.

Question 5: CRITICAL THINKING

Two students analyze the same population data for a city from 1970 to 2020. Student A uses a linear model with R² = 0.88 and randomly scattered residuals. Student B uses a 6th-degree polynomial with R² = 0.999 and very small residuals. Student B claims her model is clearly superior because R² is nearly perfect. Write a paragraph explaining why Student A's linear model might actually be the better choice, and identify the key risk in Student B's approach.

Show answer

Student B's 6th-degree polynomial is almost certainly overfitting the data. A 6th-degree polynomial has 7 parameters and can pass very close to a small number of data points purely by flexing through each one, without capturing the actual underlying trend. If you were to use Student B's model to predict the city's population in 2030, it would likely produce wildly unrealistic values — 6th-degree polynomials tend to shoot toward positive or negative infinity rapidly outside the data range. Student A's linear model, while less flashy with R² = 0.88, has randomly scattered residuals confirming that a linear trend is appropriate for the data. Its predictions beyond 2020 would be more stable and reasonable. This illustrates the principle of parsimony: the simplest model that adequately fits the data is usually the most trustworthy for prediction. The key risk in Student B's approach is overfitting — mistaking noise for signal and creating a model that memorizes the data rather than learning from it.

SUMMARY

Lesson Summary

Summary

Comparing models means going beyond gut feeling to use quantitative criteria and contextual reasoning. Start by computing the R² value for each model to see how much variation each explains. Then calculate residuals and examine them in a residual plot — random scatter means the model type is appropriate, while a curved or fanning pattern means it is not.

Always check reasonableness: does the model give sensible predictions within and slightly beyond the data range? When two models fit similarly, apply the principle of parsimony and prefer the simpler one. A complete justification cites multiple criteria — R², SSR, residual patterns, and real-world context — never just one number. These skills form the foundation for advanced techniques like adjusted R², cross-validation, and information criteria that you will encounter in future statistics and data science courses.