Comparing Model Fits — I can compare two models (linear vs quadratic) for a data set and justify which fits better conceptually. - Math 2

SECTION 1

Historical Context & Motivation

For centuries, scientists and mathematicians have tried to describe patterns in data using equations. The simplest approach — drawing a straight line — works remarkably well for many situations, but the real world doesn't always follow a straight path. A ball tossed into the air, the spread of a rumor through a school, or the braking distance of a car at different speeds all trace curves rather than lines. Recognizing when a line is adequate and when a curve is needed has been a driving question in statistics since its earliest days.

1805 • Legendre & Least Squares

Adrien-Marie Legendre published the method of least squares, giving mathematicians a systematic way to fit a line to data by minimizing the sum of squared errors.

1809 • Gauss Extends the Method

Carl Friedrich Gauss showed that least-squares fitting could be applied to curves and polynomials, not just straight lines, opening the door to quadratic regression and beyond.

1885 • Galton & Regression

Francis Galton coined the term regression while studying heredity, formalizing the concept of fitting models to bivariate data.

1920s • R² Emerges

Statisticians developed the coefficient of determination (R²) as a standard measure for how well a model explains variation in data, making it possible to objectively compare competing models.

Today • Technology & Model Selection

Graphing calculators and software instantly compute multiple regressions and residual plots, allowing students and professionals to compare linear, quadratic, and other models in seconds.

The central question this lesson addresses is deceptively simple: given a set of data, should you model it with a line or a parabola, and how do you defend that choice? Answering this requires both mathematical tools (like R² and residual plots) and conceptual reasoning about what the data actually represents.

SECTION 2

Core Principles & Definitions

Before you can compare two models, you need to understand the vocabulary and the key ideas behind model fitting. A model is simply an equation that attempts to describe the relationship between two variables. When we say a model "fits" data well, we mean its predicted values are consistently close to the actual data points. The following principles form the foundation of model comparison.

Linear Model

A linear model has the form y = ax + b. It predicts a constant rate of change — every time x increases by 1, y changes by the same amount a.

Quadratic Model

A quadratic model has the form y = ax² + bx + c. It captures situations where the rate of change itself is changing — the data curves up or down.

Residuals

A residual is the difference between an observed value and the value predicted by the model: residual = observed − predicted. Residuals tell you how far off the model is at each data point.

Residual Plot

A residual plot graphs residuals on the y-axis against x-values on the x-axis. A good fit shows residuals scattered randomly around zero; a pattern signals the model is missing something.

R² (Coefficient of Determination)

R² measures the proportion of variation in y that is explained by the model. It ranges from 0 to 1 — the closer to 1, the better the fit. Comparing R² values across models is one quick way to judge improvement.

✦

KEY TAKEAWAY

Think of fitting a model like choosing shoes for a sport. A pair of basic sneakers (a linear model) works fine for walking, but if you need to play basketball (your data curves), you'll perform much better with basketball shoes (a quadratic model). The key is matching the tool to the task — and the residual plot is like trying them on to see which actually fits your feet.

SECTION 3

Visual Explanation — Seeing the Difference

The most powerful way to understand model comparison is to look at the same data with two different models overlaid on it, and then examine the residual plots side by side. The diagram below shows a scatter plot of braking distance versus car speed. A linear fit is shown in blue, and a quadratic fit is shown in violet. Notice how the linear model systematically misses the data at low and high speeds.

The scatter plot shows braking distance (ft) versus speed (mph). The dashed blue line is the linear fit; the solid violet curve is the quadratic fit. Notice how the quadratic curve passes much closer to the data points, especially at low and high speeds where the linear model consistently overshoots or undershoots.

In the diagram above, the linear model appears to underpredict braking distance at both low and high speeds while overpredicting in the middle range. This U-shaped pattern of error is a classic sign that the relationship between the variables is curved, not straight. The quadratic model, by contrast, hugs the data much more tightly because it accounts for the increasing rate at which braking distance grows as speed rises.

SECTION 4

Mathematical Framework

Two mathematical tools are essential for comparing model fits: the residual and the coefficient of determination (R²). Understanding how each is calculated will help you use them with confidence.

LINEAR MODEL

y = ax + b

a = slope (rate of change), b = y-intercept (starting value). This model assumes a constant rate of change.

QUADRATIC MODEL

y = ax² + bx + c

a = curvature coefficient (determines whether the parabola opens up or down), b = linear coefficient, c = y-intercept. This model allows the rate of change to increase or decrease over time.

RESIDUAL

eᵢ = yᵢ − ŷᵢ

eᵢ = residual for the i-th data point, yᵢ = observed (actual) value, ŷᵢ = predicted value from the model. A positive residual means the model underpredicted; a negative residual means it overpredicted.

COEFFICIENT OF DETERMINATION

R² = 1 − (SS_res / SS_tot)

SS_res = sum of squared residuals = Σ(yᵢ − ŷᵢ)². SS_tot = total sum of squares = Σ(yᵢ − ȳ)². R² close to 1 means the model explains nearly all the variation in the data.

⚠

Important Note on R²

A higher R² does not automatically mean a model is better. Adding more terms to a model will almost always increase R², even if the added complexity is not justified. That's why you should always check the residual plot and consider whether the model makes sense for the situation. R² is a useful number, but it's only one piece of the puzzle.

SECTION 5

Residual Plots — The Deciding Factor

The single most important visual tool for comparing model fits is the residual plot. When a model fits the data well, its residuals should look like a random scatter of points above and below the horizontal line at zero — no pattern, no trend, no curve. When you see a clear pattern in the residual plot, it means your model is systematically missing something. The diagram below shows two residual plots for the same data: one from a linear model and one from a quadratic model.

Left: the residual plot for the linear model shows a clear U-shaped pattern — the model systematically under- and over-predicts. Right: the residual plot for the quadratic model shows points scattered randomly around zero — indicating a much better fit.

The left residual plot reveals a telltale U-shaped pattern: the residuals are positive at the ends and negative in the middle. This curved pattern is exactly what you expect when you force a straight line onto data that is actually curved. The quadratic residual plot on the right, by contrast, shows no systematic pattern — the points bounce above and below zero in an essentially random way, which is exactly what a well-fitting model should produce.

✓ Pattern in residuals → The model is missing a feature of the data. Try a different model.
✓ Random scatter in residuals → The model captures the main trend. The fit is appropriate.
✓ Increasing spread (fan shape) → The model may fit the trend, but predictions become less reliable for larger x-values.

SECTION 6

Worked Example — Choosing the Better Model

A physics class measures the height of a ball (in feet) at various times (in seconds) after it is launched upward. The data are shown below. Two regressions are computed: a linear model and a quadratic model. Let's walk through how to decide which model fits better.

Time (s)	Height (ft)
0	4
0.5	20
1.0	28
1.5	30
2.0	24
2.5	12
3.0	0

Ball launch height data

Comparing Linear and Quadratic Models for Ball Height Data

1
Step 1 — Fit a Linear Model

Using a calculator or regression tool, the linear model is y = −1.14x + 20.57. The R² value for this model is R² = 0.244. This means the line explains only about 24.4% of the variation in height.

Linear: y = −1.14x + 20.57, R² ≈ 0.244
2
Step 2 — Fit a Quadratic Model

The quadratic model is y = −16x² + 40x + 4. The R² value is R² = 0.994. This means the parabola explains about 99.4% of the variation — a dramatic improvement.

Quadratic: y = −16x² + 40x + 4, R² ≈ 0.994
3
Step 3 — Examine the Residual Plots

The linear residual plot shows a strong curved (inverted-U) pattern: positive residuals in the middle, negative at the ends. The quadratic residual plot shows residuals scattered close to zero with no discernible pattern. This confirms the quadratic model is capturing the trend that the linear model misses.

Linear residuals: curved pattern. Quadratic residuals: random scatter.
4
Step 4 — Apply Conceptual Reasoning

Physics tells us that the height of a projectile under gravity follows the equation h = −½gt² + v₀t + h₀, which is a quadratic function of time. The situation itself — an object launched upward and pulled back down by gravity — demands a model with a squared term. A straight line cannot capture the rise-then-fall behavior.

Context confirms: projectile motion is fundamentally quadratic.
5
Step 5 — State Your Conclusion

The quadratic model y = −16x² + 40x + 4 fits the data far better than the linear model. Evidence: R² jumps from 0.244 to 0.994, the residual plot becomes patternless, and the real-world context of projectile motion requires a quadratic relationship.

Conclusion: The quadratic model is the better fit — supported by R², residual analysis, and real-world reasoning.

SECTION 7

Strengths & Limitations of Each Model Type

Neither the linear model nor the quadratic model is universally "better." Each has strengths in certain situations and weaknesses in others. The table below summarizes when to reach for each.

Feature	Linear Model	Quadratic Model
Shape of fit	Straight line	Parabola (U or inverted U)
Best when data shows…	Constant rate of change	Changing rate of change (acceleration or deceleration)
Number of parameters	2 (slope, intercept)	3 (a, b, c)
Extrapolation risk	Moderate — continues in same direction	High — parabola may curve wildly outside the data range
Simplicity	Simple and easy to interpret	Slightly more complex; vertex and direction matter
Common real-world examples	Cost per item, hourly wages, temperature rising at a constant rate	Projectile motion, profit optimization, braking distance

Comparison of linear and quadratic models

✦

KEY TAKEAWAY

Think of the linear model as a ruler — great for measuring straight edges, but useless for tracing the curved rim of a bowl. The quadratic model is like a flexible curve tool that can bend to match the shape of the data. Always start with the simpler model (the ruler), and only switch to the curve if the residual plot tells you the straight edge isn't working. Simplicity is a virtue — don't use a quadratic model if a line already fits well.

SECTION 8

Connection to Advanced Model Fitting

The skills you are building now — comparing two models using residuals, R², and conceptual reasoning — are the same skills used in advanced statistics and data science. In future courses, you will encounter exponential, logarithmic, and sinusoidal models, and you will use more sophisticated criteria to choose among them. The table below previews how the tools you know now extend into higher-level work.

Concept Now	Advanced Version
R² to compare two models	Adjusted R² — penalizes adding unnecessary parameters so you can fairly compare models with different numbers of terms
Residual plot (visual check)	Formal residual diagnostics — statistical tests for normality, homoscedasticity, and independence of residuals
Linear vs quadratic	Multiple regression, polynomial regression, and non-linear models (exponential, logistic, etc.)
Conceptual justification	Model selection criteria such as AIC and BIC, which balance fit quality with model complexity

Current tools vs advanced extensions

The core philosophy never changes, no matter how advanced the statistics get: the best model is the simplest one that adequately captures the pattern in the data and makes sense in context. This idea is sometimes called the principle of parsimony, and it will guide your work in AP Statistics, college-level courses, and professional data analysis.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

A student fits a linear model to a data set and produces a residual plot that shows a clear U-shaped curve. What does this pattern suggest, and what should the student try next?

Show answer

A U-shaped pattern in the residual plot means the linear model is systematically missing part of the data's trend — it underpredicts at the extremes and overpredicts in the middle (or vice versa). This is a strong signal that the underlying relationship is curved rather than straight. The student should try fitting a quadratic model and then re-examine the residual plot. If the new residual plot shows a random scatter of points around zero, the quadratic model is a better fit.

Question 2: BASIC CALCULATION

A linear model gives R² = 0.72 and a quadratic model gives R² = 0.96 for the same data set. Which model explains more variation? By how many percentage points does the better model improve on the weaker one?

Show answer

The quadratic model explains more variation because its R² (0.96) is higher than the linear model's R² (0.72). In terms of percentage points, the improvement is 0.96 − 0.72 = 0.24, or 24 percentage points. The linear model explains 72% of the variation in y, while the quadratic model explains 96%. This is a substantial improvement and suggests the quadratic model is capturing a curvature in the data that the line misses.

Question 3: INTERMEDIATE

A data set of hours studied (x) versus test score (y) produces the following results: Linear model: y = 5.2x + 62, R² = 0.91, residual plot shows random scatter. Quadratic model: y = −0.1x² + 6.0x + 60, R² = 0.92, residual plot also shows random scatter. Which model would you choose and why?

Show answer

Even though the quadratic model has a slightly higher R² (0.92 vs 0.91), you should choose the linear model. Both residual plots show random scatter, meaning both models fit the data adequately. The improvement of 0.01 in R² is negligible. Since the linear model is simpler — it has fewer parameters and is easier to interpret — it is preferred by the principle of parsimony. Additionally, a roughly constant gain in test score per hour of study is conceptually reasonable, so the linear model aligns well with the real-world context.

Question 4: APPLIED

A small business tracks monthly revenue (in thousands of dollars) over its first 8 months. Revenue starts low, grows quickly, then begins to level off. A linear model gives R² = 0.84 with a curved residual pattern. A quadratic model gives R² = 0.97 with random residuals. Explain which model is better and what the quadratic model tells the business owner about revenue growth.

Show answer

The quadratic model is the better choice. It has a much higher R² (0.97 vs 0.84), and critically, its residual plot is random while the linear residual plot shows a pattern — indicating the linear model is systematically misrepresenting the trend. The quadratic model's negative leading coefficient (since revenue levels off) tells the business owner that revenue growth is decelerating over time. Each additional month brings less new revenue than the month before. This is a realistic scenario for a new business entering market saturation. The owner should be cautious about extrapolating the quadratic model too far, since a parabola eventually turns downward, which may or may not reflect reality.

Question 5: CRITICAL THINKING

A researcher collects data on the relationship between temperature (°F) and the number of ice cream cones sold per day. She fits both a linear and a quadratic model. The quadratic model has a slightly higher R² (0.89 vs 0.87), but the linear residual plot shows random scatter. A colleague argues the quadratic model is better because it has a higher R². Write a reasoned response evaluating the colleague's claim.

Show answer

The colleague's reasoning is flawed. R² alone is not sufficient to declare a model superior — context and residual analysis matter more. Since the linear residual plot already shows random scatter, the linear model is capturing the main trend effectively. The small R² increase (0.02) from the quadratic model is likely due to the extra parameter fitting noise rather than a genuine curvature in the data. Furthermore, there is no obvious conceptual reason why ice cream sales would accelerate and then decelerate with temperature in a parabolic fashion; a roughly constant increase per degree is reasonable. Adding unnecessary complexity violates the principle of parsimony. Unless the quadratic residual plot reveals a meaningfully different pattern or the researcher has a strong theoretical reason to expect curvature, the linear model should be preferred.

SUMMARY

Lesson Summary

Summary

When deciding between a linear model (y = ax + b) and a quadratic model (y = ax² + bx + c), use three complementary tools. First, compare R² values — the model with the higher R² explains more of the variation in the data. Second, examine the residual plots — a random scatter around zero signals a good fit, while a U-shaped or curved pattern signals the model is missing curvature in the data. Third, apply conceptual reasoning — ask whether the real-world situation logically involves a constant rate of change (linear) or a changing rate of change (quadratic).

Always start with the simpler linear model and only move to the quadratic model if the data, the residuals, and the context all point toward curvature. A tiny R² improvement with no residual pattern change does not justify added complexity. The guiding principle is parsimony — choose the simplest model that adequately captures the trend and makes sense in the real world.