01 / 21

Linear Regression
Models

Learn to model relationships between variables using the least-squares regression line, interpret slope and intercept, and assess how well a line fits data.

AP Statistics

What Is Linear Regression?

Linear regression is a statistical method that finds the best-fitting straight line through a set of data points. It models the relationship between an explanatory variable \( x \) and a response variable \( y \).

Explanatory Variable
The independent variable \( x \) that we use to predict or explain changes in the response.
Response Variable
The dependent variable \( y \) that we are trying to predict or model.

Scatterplots and Correlation

Before fitting a regression line, always start with a scatterplot. This lets you check whether a linear model is appropriate by examining the direction, form, and strength of the association.

Study Hours (x) Exam Score (y) 0 2 4 6 8 10 50 60 70 80 90 LSRL

Why Use Regression?

Linear regression lets us do more than just describe a relationship. It gives us a predictive model we can use to estimate values of \( y \) for new values of \( x \).

Prediction
Estimate the response for a given explanatory value using the regression equation.
Quantify
The slope \( b \) tells us the average change in \( y \) per one-unit increase in \( x \).
Assess Fit
Use \( r^2 \) to determine the proportion of variability in \( y \) explained by \( x \).

The Least-Squares Regression Line

The least-squares regression line (LSRL) minimizes the sum of squared residuals: the vertical distances from each data point to the line.

Slope \( b \)
\( b = r \cdot \dfrac{s_y}{s_x} \) : the predicted change in \( y \) per unit increase in \( x \).
Intercept \( a \)
\( a = \bar{y} - b\bar{x} \) : the predicted value of \( y \) when \( x = 0 \).

Understanding Residuals

A residual is the difference between the observed value and the predicted value: \( e = y - \hat{y} \). Positive residuals are above the line; negative residuals are below.

x y +ve −ve 0 LSRL

\( r \) vs. \( r^2 \): Key Distinction

Students often confuse the correlation coefficient \( r \) with the coefficient of determination \( r^2 \). They measure different things.

\( r \) (Correlation) \( r^2 \) (Determination)
Range −1 to +1 0 to 1
Measures Strength and direction Proportion of variability explained
Sign Positive or negative Always non-negative
Interpretation "Strong negative linear association" "82% of variability in \( y \) is explained by \( x \)"

Interpreting Slope and Intercept

On the AP exam, you must interpret slope and intercept in context. Use the template below as a guide.

1
Slope \( b \): "For each additional [unit of \( x \)], the predicted [response \( y \)] changes by \( b \) [units of \( y \)], on average."
2
Intercept \( a \): "When [\( x \)] is 0, the predicted [\( y \)] is \( a \) [units]." Note: this may not always have a practical meaning.
3
\( r^2 \): "Approximately [value]% of the variability in [\( y \)] is explained by the linear relationship with [\( x \)]."

How to Build a Regression Model

1
Make a scatterplot. Check that the relationship looks approximately linear with no strong curvature or outliers.
2
Calculate \( r \). Confirm a reasonably strong linear association (typically \( |r| \geq 0.7 \)).
3
Find the LSRL. Compute \( b = r \cdot \frac{s_y}{s_x} \) and \( a = \bar{y} - b\bar{x} \). Write the equation \( \hat{y} = a + bx \).
4
Check the residual plot. Plot residuals vs. \( x \). Look for random scatter (no pattern) to confirm the linear model is appropriate.
5
Interpret in context. State slope, intercept, and \( r^2 \) using the language of the problem.

Reading Residual Plots

A residual plot graphs residuals \( (y - \hat{y}) \) against \( x \). A good residual plot shows random scatter around zero. A bad one shows a clear pattern (curved, fanning, etc.).

✓ Good: Random Scatter
Points scattered randomly above and below 0 with no pattern. Linear model is appropriate.
✕ Bad: Curved Pattern
A U-shape or arch suggests the data are not linear. Consider a transformation.
✕ Bad: Fanning
Residuals spread out (or narrow) as \( x \) increases. This violates equal variance.

Example 1: Finding the LSRL

A study records the number of hours studied (\( x \)) and exam scores (\( y \)) for 8 students. Summary statistics:

\( \bar{x} = 5 \)
Mean hours studied
\( \bar{y} = 75 \)
Mean exam score
\( s_x = 2 \)
SD of hours
\( s_y = 10 \)
SD of scores

Example 1: Interpretation

Now interpret each part of \( \hat{y} = 53.75 + 4.25x \) in the context of the problem.

b
Slope = 4.25: For each additional hour studied, the predicted exam score increases by 4.25 points, on average.
a
Intercept = 53.75: A student who studies 0 hours is predicted to score 53.75. (This is the model's prediction; may not be practical.)
3
\( r^2 = 0.7225 \): Approximately 72.25% of the variability in exam scores is explained by the linear relationship with hours studied.

Example 2: Computing a Residual

Using the model \( \hat{y} = 53.75 + 4.25x \), suppose a student who studied 6 hours actually scored 82.

1
Predicted value: \( \hat{y} = 53.75 + 4.25(6) = 53.75 + 25.5 = 79.25 \)
2
Residual: \( e = y - \hat{y} = 82 - 79.25 = \) +2.75
3
Interpretation: The actual score was 2.75 points higher than the model predicted. The point lies above the LSRL.

Extrapolation and Influential Points

Extrapolation means using the regression line to predict beyond the range of observed \( x \)-values. This is unreliable because we have no data to support the model in that region.

x Data range (safe) Extrapolation zone ?

Common Mistakes

✕ Forgetting to Square \( r \) for \( r^2 \)
A common error is to use \( r \) instead of \( r^2 \). For example, if \( r = 0.9 \), it's wrong to say "90% of variability is explained." You must square it: \( r^2 = (0.9)^2 = 0.81 \), so 81% is explained.
✕ Extrapolating Beyond Data
Predicting far outside the observed \( x \)-range. The linear pattern may not hold beyond your data.
✕ Correlation is Not Causation
A strong \( r \) does not mean \( x \) causes changes in \( y \). Lurking variables or coincidence may be at play.
✕ Incorrect Residual Formula
Writing \( \hat{y} - y \) instead of \( y - \hat{y} \). Residual = Observed minus Predicted, not the reverse.
✕ Ignoring Context
Interpreting slope as just "4.25" without naming the variables. AP graders require full contextual interpretation.

Summary: Key Formulas and Concepts

Concept Formula / Fact Key Detail
LSRL Equation \( \hat{y} = a + bx \) Minimizes \( \sum(y_i - \hat{y}_i)^2 \)
Slope \( b = r \cdot \frac{s_y}{s_x} \) Change in predicted \( y \) per unit \( x \)
Intercept \( a = \bar{y} - b\bar{x} \) Predicted \( y \) when \( x = 0 \)
Correlation \( r \) −1 to +1 Strength and direction of linear association
\( r^2 \) 0 to 1 Proportion of variability explained
Residual \( e = y - \hat{y} \) Observed − Predicted

Question 1 of 5

A regression line has the equation \( \hat{y} = 12.4 + 3.2x \). Which of the following is the best interpretation of the slope?

Question 2 of 5

If \( r = -0.8 \), what is the value of \( r^2 \) and what does it mean?

Question 3 of 5

A student uses the model \( \hat{y} = 20 + 5x \) and finds that the actual value at \( x = 4 \) is \( y = 38 \). What is the residual?

Question 4 of 5

A residual plot shows a clear U-shaped pattern. What does this indicate?

Question 5 of 5

Given \( \bar{x} = 10 \), \( \bar{y} = 50 \), \( s_x = 4 \), \( s_y = 8 \), and \( r = 0.6 \), what is the equation of the LSRL?