Historical Context & Motivation
Throughout history, scientists and mathematicians have faced a recurring challenge: when multiple equations can describe the same phenomenon, how do you decide which one to trust? The practice of model comparison has roots stretching back centuries, arising naturally whenever people tried to predict eclipses, population growth, or economic trends. The core question has always been the same: does this equation actually capture what is happening in the real world, or is there a better one?
Today, model comparison is everywhere — from choosing between a linear and exponential model for COVID case data to deciding whether a quadratic or logarithmic curve best predicts a company's revenue. The central question this lesson addresses is: given the same data set, how do you rigorously compare two models and justify which is better?
Core Principles of Model Comparison
Before you can compare two models, you need a shared vocabulary and a set of criteria. Model comparison rests on several foundational ideas that guide you from raw data to a justified conclusion. Understanding these principles will prevent you from simply picking whichever equation "looks right" and instead give you tools to defend your choice with evidence.
Residuals
Coefficient of Determination (R²)
Residual Patterns
Reasonableness
Parsimony (Simplicity)
Visualizing Two Models on the Same Data
The most powerful first step in comparing models is to graph them both on the same set of axes alongside the original data. The diagram below shows a scatter plot of data points representing the number of subscribers to a streaming service over its first eight months, with both a linear model and an exponential model overlaid.
From the graph alone you can see that the linear model consistently misses the data in the later months — it predicts too few subscribers at month 7 and too many at month 1. The exponential curve tracks the upward acceleration more faithfully. But visual inspection is only the beginning; in the next sections you will learn how to quantify this difference.
Mathematical Framework for Model Comparison
Visuals help build intuition, but a rigorous comparison requires numbers. The two primary quantitative tools are residual analysis and the coefficient of determination. Both give you concrete evidence you can cite when justifying your model choice.
Residual Plots — Reading the Pattern
A residual plot graphs the residuals (vertical axis) against the independent variable or predicted values (horizontal axis). If the model is appropriate, residuals should look like a random cloud of points centered on zero. A clear curve or pattern tells you the model is systematically missing something. The diagram below compares the residual plots for the linear and exponential models from Section 3.
The curved pattern on the left is the smoking gun. Even though the linear model might have a decent R² value, the residual plot reveals that a linear equation is the wrong type of model for this data. The random scatter on the right confirms that the exponential model captures the underlying trend without systematic error.
- Random scatter around zero → the model type is appropriate.
- Curved pattern → try a different model type (e.g., switch from linear to quadratic or exponential).
- Fanning pattern (residuals grow larger as x increases) → the model's accuracy varies across the data range and may need a transformation.
Worked Example — Choosing the Better Model
A biology class measured the area (cm²) of a bacterial colony over 6 hours. Two regression models were generated: a linear model y = 3.2x + 0.5 (R² = 0.91) and a quadratic model y = 0.4x² + 0.8x + 1.0 (R² = 0.98). Which model is better, and why?
| Hour (x) | Observed Area (y) | Linear ŷ | Quadratic ŷ |
|---|---|---|---|
| 0 | 1.0 | 0.5 | 1.0 |
| 1 | 2.1 | 3.7 | 2.2 |
| 2 | 3.8 | 6.9 | 4.2 |
| 3 | 7.0 | 10.1 | 7.0 |
| 4 | 10.5 | 13.3 | 10.6 |
| 5 | 15.2 | 16.5 | 15.0 |
| 6 | 20.8 | 19.7 | 20.2 |
Comparison Criteria — Strengths & Limitations
No single criterion tells the whole story. The table below summarizes the tools you have for comparing models, along with what each one does well and where it falls short. A strong justification references multiple criteria rather than relying on just one.
| Criterion | Strength | Limitation |
|---|---|---|
| R² value | Easy to compute and interpret; gives a quick sense of overall fit quality. | Always increases (or stays the same) when you add more terms, so it can reward overly complex models. |
| SSR | Directly measures total prediction error; lower is always better for the same data set. | Hard to interpret in isolation without context — a SSR of 10 could be great or terrible depending on the data scale. |
| Residual plot | Reveals whether the model type is appropriate; detects patterns that R² alone cannot. | Subjective — what counts as a "pattern" versus random noise can be debatable with small data sets. |
| Reasonableness | Ensures the model makes real-world sense; catches models that fit data but produce absurd predictions. | Requires domain knowledge — you need to understand the context to judge what counts as "reasonable." |
| Parsimony | Guards against overfitting; simpler models tend to generalize better to new data. | Sometimes a more complex model is genuinely needed — simplicity for its own sake can mean poor fit. |
Connection to Advanced Model Selection
The comparison techniques you have learned — R², residual analysis, and reasonableness checks — form the foundation for more sophisticated methods used in statistics, data science, and machine learning. As you progress in mathematics, you'll encounter tools that formalize the trade-off between fit and complexity.
| What You Know Now | Where It Leads |
|---|---|
| R² — measures explained variation | Adjusted R² — penalizes adding unnecessary variables so you can compare models with different numbers of terms fairly |
| Visual residual plots — look for patterns | Statistical tests (e.g., Durbin-Watson test) that quantify whether residual patterns are statistically significant |
| Parsimony — prefer simpler models | AIC and BIC — information criteria that mathematically balance fit against the number of parameters |
| Reasonableness — check predictions | Cross-validation — splitting data into training and test sets to see how well the model predicts unseen values |
The key insight that carries forward is this: a model is only as good as its predictions. Whether you're fitting a line in Grade 11 math or training a neural network in college, the fundamental question remains: does this model capture the real pattern in the data, or is it just memorizing noise? The skills you're building now — checking fit, spotting patterns in residuals, and asking whether predictions make sense — are exactly the skills that professional data scientists use every day.
Practice Problems
Lesson Summary
Comparing models means going beyond gut feeling to use quantitative criteria and contextual reasoning. Start by computing the R² value for each model to see how much variation each explains. Then calculate residuals and examine them in a residual plot — random scatter means the model type is appropriate, while a curved or fanning pattern means it is not.
Always check reasonableness: does the model give sensible predictions within and slightly beyond the data range? When two models fit similarly, apply the principle of parsimony and prefer the simpler one. A complete justification cites multiple criteria — R², SSR, residual patterns, and real-world context — never just one number. These skills form the foundation for advanced techniques like adjusted R², cross-validation, and information criteria that you will encounter in future statistics and data science courses.