Linear Regression
Models
Learn to model relationships between variables using the least-squares regression line, interpret slope and intercept, and assess how well a line fits data.
What Is Linear Regression?
Linear regression is a statistical method that finds the best-fitting straight line through a set of data points. It models the relationship between an explanatory variable \( x \) and a response variable \( y \).
Scatterplots and Correlation
Before fitting a regression line, always start with a scatterplot. This lets you check whether a linear model is appropriate by examining the direction, form, and strength of the association.
Why Use Regression?
Linear regression lets us do more than just describe a relationship. It gives us a predictive model we can use to estimate values of \( y \) for new values of \( x \).
The Least-Squares Regression Line
The least-squares regression line (LSRL) minimizes the sum of squared residuals: the vertical distances from each data point to the line.
Understanding Residuals
A residual is the difference between the observed value and the predicted value: \( e = y - \hat{y} \). Positive residuals are above the line; negative residuals are below.
\( r \) vs. \( r^2 \): Key Distinction
Students often confuse the correlation coefficient \( r \) with the coefficient of determination \( r^2 \). They measure different things.
| \( r \) (Correlation) | \( r^2 \) (Determination) | |
|---|---|---|
| Range | −1 to +1 | 0 to 1 |
| Measures | Strength and direction | Proportion of variability explained |
| Sign | Positive or negative | Always non-negative |
| Interpretation | "Strong negative linear association" | "82% of variability in \( y \) is explained by \( x \)" |
Interpreting Slope and Intercept
On the AP exam, you must interpret slope and intercept in context. Use the template below as a guide.
How to Build a Regression Model
Reading Residual Plots
A residual plot graphs residuals \( (y - \hat{y}) \) against \( x \). A good residual plot shows random scatter around zero. A bad one shows a clear pattern (curved, fanning, etc.).
Example 1: Finding the LSRL
A study records the number of hours studied (\( x \)) and exam scores (\( y \)) for 8 students. Summary statistics:
Example 1: Interpretation
Now interpret each part of \( \hat{y} = 53.75 + 4.25x \) in the context of the problem.
Example 2: Computing a Residual
Using the model \( \hat{y} = 53.75 + 4.25x \), suppose a student who studied 6 hours actually scored 82.
Extrapolation and Influential Points
Extrapolation means using the regression line to predict beyond the range of observed \( x \)-values. This is unreliable because we have no data to support the model in that region.
Common Mistakes
Summary: Key Formulas and Concepts
| Concept | Formula / Fact | Key Detail |
|---|---|---|
| LSRL Equation | \( \hat{y} = a + bx \) | Minimizes \( \sum(y_i - \hat{y}_i)^2 \) |
| Slope | \( b = r \cdot \frac{s_y}{s_x} \) | Change in predicted \( y \) per unit \( x \) |
| Intercept | \( a = \bar{y} - b\bar{x} \) | Predicted \( y \) when \( x = 0 \) |
| Correlation \( r \) | −1 to +1 | Strength and direction of linear association |
| \( r^2 \) | 0 to 1 | Proportion of variability explained |
| Residual | \( e = y - \hat{y} \) | Observed − Predicted |
Question 1 of 5
A regression line has the equation \( \hat{y} = 12.4 + 3.2x \). Which of the following is the best interpretation of the slope?
Question 2 of 5
If \( r = -0.8 \), what is the value of \( r^2 \) and what does it mean?
Question 3 of 5
A student uses the model \( \hat{y} = 20 + 5x \) and finds that the actual value at \( x = 4 \) is \( y = 38 \). What is the residual?
Question 4 of 5
A residual plot shows a clear U-shaped pattern. What does this indicate?
Question 5 of 5
Given \( \bar{x} = 10 \), \( \bar{y} = 50 \), \( s_x = 4 \), \( s_y = 8 \), and \( r = 0.6 \), what is the equation of the LSRL?