01 / 21

Competing Function
Model Validation

Learn to choose the best-fit function model by analyzing residuals, comparing predicted vs. observed values, and evaluating goodness of fit.

AP Precalculus

What Is Model Validation?

When we fit a function to data, we need to check: how well does the model actually represent the data? Model validation is the process of testing a proposed function against observed values to determine if it is a reliable predictor.

Why Competing Models?

Real data rarely tells you which function type to use. The same dataset might plausibly be modeled by a linear, quadratic, or exponential function. We compare competing models to find the one that best captures the underlying behavior.

Linear
\( f(x) = ax + b \): constant rate of change
Quadratic
\( f(x) = ax^2 + bx + c \): changing rate of change
Exponential
\( f(x) = ab^x \): proportional rate of change

The Residual: Your Validation Tool

A residual measures the vertical distance between a data point and the model's prediction. Positive residuals mean the model underestimates; negative residuals mean it overestimates.

x y model -15 +5 -25 +15 -2 observed predicted

A cell culture where more cells create more growth events

A researcher looks through a microscope beside a closed culture vessel,sample containers and an illustrative cell inset.For the six observations 5, 10, 20, 39, 80, and 158 thousand, the model N = 5×2^t gives residuals 0, 0, 0, −1, 0, and −2 thousand.

Competing model validation data

For the six observations 5, 10, 20, 39, 80, and 158 thousand, the model N = 5×2^t gives residuals 0, 0, 0, −1, 0, and −2 thousand.For the six observations 5, 10, 20, 39, 80, and 158 thousand, the model N = 5×2^t gives residuals 0, 0, 0, −1, 0, and −2 thousand.

Residual Plots Reveal Model Fit

A residual plot graphs residuals vs. \( x \). The pattern tells you whether your model captures the data's structure:

✓ Good Fit
Residuals scatter randomly around zero with no visible pattern
✕ Poor Fit
Residuals show a curved pattern: the model misses systematic behavior
Random (Good) 0 Curved (Poor) 0

The SSR Criterion

The SSR quantifies total model error. For competing models fit to the same data, the model with the smaller SSR fits better. We square residuals so positive and negative errors don't cancel out.

Lower SSR
Predictions are closer to actual data on average
Higher SSR
Model misses the data by larger amounts overall

Measuring Fit with \( R^2 \)

\( R^2 \) tells you what fraction of the data's variability the model explains. It ranges from 0 (no explanatory power) to 1 (perfect fit). When comparing models: higher \( R^2 \) = better fit.

0 0.5 1.0 Poor Moderate Excellent

Overfitting vs. Underfitting

A model with more parameters will always have a higher \( R^2 \) on the training data, but it may be capturing noise rather than the true pattern. The best model balances accuracy with simplicity.

Underfitting
Model is too simple: misses real patterns. Residual plot shows a clear curve. Low \( R^2 \).
Good Fit
Model captures the pattern without chasing noise. Random residuals. High \( R^2 \).
Overfitting
Model is too complex: fits noise. Perfect on training data, poor on new data.

How to Validate Competing Models

1
Fit each candidate model (linear, quadratic, exponential, etc.) to the data.
2
Compute residuals \( e_i = y_i - \hat{y}_i \) for each model.
3
Plot residuals vs. \( x \). Look for patterns: random scatter = good; curved pattern = bad.
4
Compare SSR or \( R^2 \) values across models. Smaller SSR (or higher \( R^2 \)) indicates better fit.
5
Choose the simplest model that has random residuals and adequate \( R^2 \). Avoid overfitting.

Example 1: Linear vs. Quadratic

A scientist measures the height (cm) of a plant over 6 weeks. Two models are proposed:

Linear
\( \hat{y} = 3x + 2 \)
Quadratic
\( \hat{y} = 0.5x^2 + 1.2x + 1.5 \)
Week cm 1 2 3 4 5 6 Linear Quadratic Data

Example 1: Residual Comparison

Week Observed Linear Resid. Quadratic Resid.
150+1.8
27-1+1.1
3110+1.4
416+2+1.7
522+5+2.0
630+10+3.3

Example 2: Exponential vs. Linear

A bacteria colony doubles roughly every hour. Data: \((0,\,50)\), \((1,\,98)\), \((2,\,205)\), \((3,\,410)\), \((4,\,800)\). Two models:

Linear
\( \hat{y} = 185x - 50 \)
Exponential
\( \hat{y} = 50 \cdot 2^x \)
Hour Pop. 0 1 2 3 4 Linear Exponential

Example 2: Verdict

Hour Observed Linear Pred. Exp. Pred.
050-5050
198135100
2205320200
3410505400
4800690800

Example 3: Three-Way Comparison

Revenue data (\$M) for a startup over 5 years: \((1,\,12)\), \((2,\,28)\), \((3,\,50)\), \((4,\,79)\), \((5,\,115)\). Which model fits best?

Model Equation SSR \( R^2 \)
Linear\( 25.3x - 15.1 \)67.50.990
Quadratic\( 4.5x^2 + 0.8x + 7.2 \)16.60.998
Exponential\( 7.32 \cdot 1.74^x \)311.50.954

Common Mistakes to Avoid

✕ Ignoring Residual Patterns
A low SSR means nothing if residuals show a systematic curve: the model is missing a pattern.
✕ Choosing by \( R^2 \) Alone
Adding more parameters always raises \( R^2 \). Compare models of similar complexity, or check residuals.
✕ Extrapolating Blindly
A model valid for \( x \in [1, 5] \) may give absurd predictions at \( x = 20 \). Always state the domain.

Summary: Validation Criteria

Criterion Good Sign Warning Sign
Residual PlotRandom scatterCurved or fanning pattern
SSRSmall relative to SSTLarge or much worse than rival
\( R^2 \)Close to 1Low or only slightly better with added complexity
Model PredictionsPhysically plausibleNegative values, divergence outside domain
SimplicityFewest parameters neededOver-parameterized: fits noise

Worked Example: Choosing Between Exponential and Quadratic

Consider the following data set collected from a biological experiment measuring the number of bacteria (in thousands) in a culture over equally spaced time intervals. t (hours) | 0 | 1 | 2 | 3 | 4 | 5 N(t) | 5 | 10 | 20 | 39 | 80 | 158 Step 1 — Compute First Differences Δy: 10 − 5 = 5, 20 − 10 = 10, 39 − 20 = 19, 80 − 39 = 41, 158 − 80 = 78. The first differences are 5, 10, 19, 41, 78—clearly not constant, so no linear function passes exactly through all six observations. No exact linear fit — first differences vary. Step 2 — Compute Consecutive Ratios r: 10/5 = 2.00, 20/10 = 2.00, 39/20 = 1.95, 80/39 ≈ 2.05, 158/80 ≈ 1.975. All ratios are approximately 2.0, suggesting an exponential model with base b ≈ 2. Ratios ≈ 2.0 — strong exponential signal. Step 3 — Check Second Differences (Competing Test) Δ²y: 10 − 5 = 5, 19 − 10 = 9, 41 − 19 = 22, 78 − 41 = 37. The second differences are 5, 9, 22, 37—these are not constant, so no quadratic function passes exactly through all six observations. No exact quadratic fit — second differences vary. Step 4 — Propose an Exponential Model and Examine Residuals Using N(0) = 5 and the near-constant ratio 2, propose N(t) = 5·2ᵗ. Predictions are 5, 10, 20, 40, 80, 160, with residuals 0, 0, 0, −1, 0, −2. This candidate fits closely over the observed interval, but six small residuals do not establish uniqueness, a patternless error process, or future accuracy. The supplemental curves 5 + 31t and 5 + 6t² are illustrative comparators, not least-squares fits. N(t) = 5·2ᵗ fits closely here; further validation is needed. VALIDATION STRATEGY

Six observed values plotted with three named model candidates. The exponential candidate follows the observations closely over the displayed interval.
The selected exponential model leaves residuals 0,0,0,−1,0,−2; a few small residuals support comparison but do not prove a model.
Three residual plots on a common vertical scale. The linear and quadratic curves are illustrative comparisons rather than fitted models.
Three residual plots on a common vertical scale. The linear and quadratic curves are illustrative comparisons rather than fitted models.

Question 1 of 5

A residual plot for a linear model shows a clear U-shaped curve. What does this suggest?

Question 2 of 5

Model A has \( R^2 = 0.92 \) and Model B has \( R^2 = 0.97 \). Model A uses 2 parameters; Model B uses 5. Which statement is most accurate?

Question 3 of 5

For data points \((1,\,10)\), \((2,\,22)\), \((3,\,35)\), a model predicts \(\hat{y} = 11x - 2\). What is the residual at \( x = 2 \)?

Question 4 of 5

An exponential model \( \hat{y} = 5 \cdot 1.5^x \) predicts a population. At \( x = 0 \), the observed value is 5. What is the SSR contribution from this point?

Question 5 of 5

Which is the best reason to reject a model even if it has a high \( R^2 \)?