Understanding
Residuals
Learn how residuals measure the gap between observed data and predicted values, and why they are essential for evaluating regression models.
What Is a Residual?
When we fit a line (or curve) to data, each point rarely falls exactly on the line. The vertical distance between an observed data point and the value predicted by the model is called a residual.
The Residual Formula
The residual for the \(i\)-th observation is:
Visualizing Residuals on a Scatterplot
Positive vs. Negative Residuals
Residuals and Least Squares
The least-squares regression line (LSRL) is chosen specifically to minimize the sum of squared residuals:
By squaring the residuals before summing, the LSRL penalizes large errors heavily. This is why the method is called "least squares": it finds the line that makes the total squared error as small as possible.
Key Properties of Residuals
The Residual Plot
A residual plot graphs residuals (\( e \)) on the vertical axis against the explanatory variable (\( x \)) or predicted values (\( \hat{y} \)) on the horizontal axis. It is the primary tool for judging whether a linear model is appropriate.
Problematic Residual Plots
Residual vs. Error Term
| Residual \( e_i \) | Error \( \varepsilon_i \) | |
|---|---|---|
| Definition | \( y_i - \hat{y}_i \) | \( y_i - (\beta_0 + \beta_1 x_i) \) |
| Based on | Sample regression line | True population line |
| Observable? | Yes: computed from data | No: theoretical quantity |
| Sum | Always \( 0 \) (for LSRL) | Expected value \( 0 \) |
| Used for | Checking model fit | Defining the model |
How to Calculate a Residual
Example 1: Computing a Residual
A study predicts test scores from hours studied. The LSRL is: $$ \hat{y} = 52 + 4.8x $$ A student studied 10 hours and scored 95.
Example 2: Interpreting a Residual Plot
Below is a residual plot from a regression of plant height on sunlight hours. What does it tell us?
Example 3: Using Residuals to Find Observed Values
A regression equation is \( \hat{y} = 20 + 3x \). At \( x = 8 \), the residual is \( +5 \). What was the actual observed value?
Common Mistakes
Summary: Residual Checklist
| Concept | Key Fact |
|---|---|
| Formula | \( e_i = y_i - \hat{y}_i \) |
| Positive residual | Model underestimates; point above line |
| Negative residual | Model overestimates; point below line |
| Sum of residuals | \( \sum e_i = 0 \) for LSRL |
| Good residual plot | Random scatter, no pattern |
| Bad residual plot | Curve or fan shape: model is inappropriate |
| \( s \) (std. dev. of residuals) | Typical size of prediction error |
Question 1 of 5
A regression line predicts \( \hat{y} = 30 + 2x \). At \( x = 10 \), the observed value is \( y = 55 \). What is the residual?
Question 2 of 5
A residual of \( e = -8 \) means the regression model:
Question 3 of 5
Which of the following is always true for the least-squares regression line?
Question 4 of 5
A residual plot shows a clear U-shaped pattern. What does this indicate?
Question 5 of 5
The regression equation is \( \hat{y} = 15 + 5x \). At \( x = 6 \), the residual is \( e = -3 \). What is the observed value of \( y \)?