Comparing Bivariate Relationships — I can compare two...

SECTION 1

Historical Context & Motivation

Humans have been studying relationships between two variables for centuries, from tracking how crop yields change with rainfall to charting how diseases spread over time. The challenge has always been the same: when you have data showing how one quantity relates to another, which linear model best describes the pattern? Two different data sets might both appear linear at first glance, but one relationship may be far stronger, steeper, or more reliable than the other. Choosing the wrong model — or failing to compare models carefully — leads to bad predictions and flawed conclusions.

1805

Least Squares Method

Adrien-Marie Legendre published the method of least squares, giving scientists a systematic way to fit a straight line to data by minimizing the sum of squared errors.

1885

Galton's Regression Line

Francis Galton introduced the concept of regression while studying the heights of parents and children, coining the phrase regression toward the mean.

1896

Pearson's Correlation Coefficient

Karl Pearson formalized the correlation coefficient r, providing a single number to measure the strength and direction of a linear relationship between two variables.

1973

Anscombe's Quartet

Statistician Francis Anscombe demonstrated four very different data sets that share nearly identical summary statistics, proving that you must visualize data — not just calculate numbers — to choose the right model.

The core question this lesson addresses is deceptively simple: given two scatter plots, each with a linear model, how do you decide which linear relationship is stronger or more appropriate? And just as importantly, how do you justify your choice with evidence rather than gut feeling? By the end of this lesson, you will be able to compare two linear bivariate relationships side by side and defend your model selection using residuals, correlation, and the shape of the data.

SECTION 2

Core Principles & Definitions

Before comparing linear models, you need to understand the vocabulary and ideas that make comparison possible. A bivariate relationship is simply the connection between two variables — one independent (x) and one dependent (y). When we plot these pairs on a coordinate plane, we get a scatter plot, which reveals the overall shape, direction, and strength of the relationship.

Linear Model

A straight-line equation (ŷ = a + bx) used to approximate a linear pattern in a scatter plot. The slope b describes the rate of change, and the y-intercept a gives the predicted value when x = 0.

Residual

The difference between an observed y-value and the y-value predicted by the model: residual = y_observed − y_predicted. Smaller residuals mean a better fit.

Correlation Coefficient (r)

A value from −1 to 1 that measures the strength and direction of a linear relationship. Values near ±1 indicate strong linearity; values near 0 indicate a weak linear association.

Coefficient of Determination (r²)

The proportion of variation in y that is explained by the linear model. An r² of 0.92 means 92% of the variability in y is accounted for by the linear relationship. Higher r² means a better-fitting linear model.

Residual Plot

A graph of residuals vs. x-values (or predicted values). A good linear model produces residuals scattered randomly around zero; a curved pattern in the residual plot signals that a linear model may not be appropriate.

✦ KEY TAKEAWAY

Think of comparing two linear models like comparing two maps of the same city drawn at different scales. Both are straight-road maps, but one might show a much more accurate picture of actual distances. The correlation coefficient r and residual plot help you judge which linear map — which linear model — most faithfully represents the real data.

SECTION 3

Visual Explanation — Seeing the Difference

The most powerful first step in comparing bivariate relationships is to look at the data. The diagram below shows two different data sets, each with a linear model overlaid. Data Set A has points clustered tightly around the line, while Data Set B has points spread much more loosely around its line. Both relationships are linear, but one is clearly stronger than the other.

Data Set A (left, purple points, cyan line): points cluster tightly around the line, indicating a strong positive linear relationship (r ≈ 0.97). Data Set B (right, orange points, pink line): points are scattered widely around the line, indicating a weak positive linear relationship (r ≈ 0.52). Both are linear, but the strength — and the reliability of predictions — differs dramatically.

Simply eyeballing the graph already suggests that Data Set A has a stronger linear relationship, but in statistics we need to go beyond visual impression. That is why we use residual plots and the correlation coefficient r to back up our judgment with numerical evidence. In the next sections, we will build the mathematical framework to do exactly that.

SECTION 4

Mathematical Framework

When comparing two linear models for different data sets, you rely on a small toolkit of formulas and checks. The most important numerical tools are the residual for each data point, the correlation coefficient (r) for measuring linear strength, and the coefficient of determination (r²) for gauging how well the linear model fits.

RESIDUAL

Residual = y_observed − y_predicted

y_observed is the actual data value. y_predicted is the value the linear model gives for the same x. A positive residual means the model underestimates; a negative residual means it overestimates.

LINEAR MODEL

ŷ = a + bx

a is the y-intercept (the predicted y when x = 0), and b is the slope (the change in y for each unit increase in x). This model is appropriate when the scatter plot shows a roughly straight-line pattern with random residuals.

CORRELATION COEFFICIENT

r = correlation between x and y (−1 ≤ r ≤ 1)

r ranges from −1 to 1. A value near +1 means a strong positive linear relationship; near −1 means strong negative; near 0 means little to no linear association. When comparing two linear relationships, the one whose |r| is closer to 1 has the stronger linear fit.

COEFFICIENT OF DETERMINATION

r² = 1 − (SS_res / SS_tot)

SS_res is the sum of squared residuals (how much error the linear model still has). SS_tot is the total sum of squares (how much total variation exists in y). For a linear model, r² = r × r. An r² closer to 1 means the linear model explains more of the variation.

⚠️ Important Nuance

A higher r² does not automatically make a linear model appropriate. You should always check the residual plot as well. If the residual plot shows a clear curved pattern, the data may not be linear at all — even if r² is reasonably high. A good linear model produces residuals that look randomly scattered around zero.

SECTION 5

Residual Plots — The Smoking Gun

The residual plot is your most reliable tool for judging whether a linear model is appropriate. After fitting a linear model, you calculate the residual for every data point, then plot those residuals on the vertical axis against the x-values on the horizontal axis. If the linear model is a good fit, the residuals will appear randomly scattered — no curves, no fans, no obvious patterns. If you see a clear curved pattern, the relationship may not actually be linear, and the linear model is missing a structural feature of the data.

Left: residuals from the strong linear model (Data Set A) scatter tightly and randomly around zero, confirming a good linear fit (r = 0.97). Right: residuals from the weak linear model (Data Set B) scatter widely around zero, confirming a poor linear fit (r = 0.52) — the linear model exists but explains very little of the variation.

In the diagram above, both models are linear, but the residual plots and correlation values reveal a dramatic difference in fit quality. Data Set A's residuals are tightly clustered around zero (r = 0.97, r² = 0.94), meaning the linear model explains 94% of the variation. Data Set B's residuals are spread widely (r = 0.52, r² = 0.27), meaning the linear model accounts for only 27% of the variation — most of the variation in y is unexplained. Both pieces of evidence — the residual plot and r — point to Data Set A having the stronger, more reliable linear relationship.

Random scatter close to zero in the residual plot → the linear model captures the underlying trend well.
Wide random scatter in the residual plot → a linear relationship may exist but is weak; predictions will be unreliable.
Curved pattern in the residual plot → the data is not linear; the linear model is systematically missing the true shape of the relationship.

SECTION 6

Worked Example

A student collected data from two different classes. In Class 1, she recorded hours of sleep (x) and quiz score (y) for eight students. In Class 2, she recorded hours of homework completed (x) and quiz score (y) for eight students. She fit a linear model to each data set using her graphing calculator. The question: which linear relationship is stronger and more appropriate for making predictions?

Sleep vs. Score (Class 1) and Homework Hours vs. Score (Class 2) with linear model predictions

Student	Class 1: Sleep (x) / Score (y)	ŷ (Class 1 Linear)	Class 2: HW Hours (x) / Score (y)	ŷ (Class 2 Linear)
1	5 hrs sleep / 62	63.1	1 hr HW / 58	55.4
2	6 hrs sleep / 68	67.8	2 hrs HW / 72	63.1
3	6 hrs sleep / 70	67.8	2 hrs HW / 65	63.1
4	7 hrs sleep / 74	72.5	3 hrs HW / 80	70.8
5	7 hrs sleep / 75	72.5	3 hrs HW / 61	70.8
6	8 hrs sleep / 79	77.2	4 hrs HW / 85	78.5
7	8 hrs sleep / 80	77.2	4 hrs HW / 70	78.5
8	9 hrs sleep / 85	81.9	5 hrs HW / 90	86.2

Comparing Two Linear Models: Sleep vs. Score and Homework vs. Score

Step 1 — Examine the Scatter Plot ShapeClass 1 scores rise steadily from 62 to 85 as sleep increases from 5 to 9 hours — the points follow a fairly tight upward band. Class 2 scores also generally rise, but with much more scatter around the trend line. Both scatter plots suggest a positive linear association, but Class 1 looks tighter.

Step 2 — Compute Residuals for Class 1 (Sleep Model)For each Class 1 data point, subtract the linear prediction from the observed score. For example, at x = 5: residual = 62 − 63.1 = −1.1. At x = 9: residual = 85 − 81.9 = 3.1. All residuals are small (within about ±3 points) and alternate between positive and negative with no obvious pattern.

Class 1 residuals are small and randomly scattered around zero.

Step 3 — Compute Residuals for Class 2 (Homework Model)Repeating the process for Class 2: at x = 1, residual = 58 − 55.4 = 2.6; at x = 2, residual = 72 − 63.1 = 8.9; at x = 3, residual = 61 − 70.8 = −9.8. These residuals vary widely in size and swing between positive and negative values, indicating the linear model's predictions are often far from the actual scores.

Class 2 residuals vary widely in magnitude and are inconsistently scattered around zero.

Step 4 — Compare r and r² ValuesThe calculator reports r = 0.97 (r² = 0.94) for the Class 1 sleep model and r = 0.72 (r² = 0.52) for the Class 2 homework model. The Class 1 linear model explains 94% of the variation in quiz scores, compared to only 52% for Class 2.

r_{Class 1} = 0.97 ≫ r_{Class 2} = 0.72

Step 5 — Justify Your ChoiceThe Class 1 (sleep vs. score) linear relationship is stronger and more appropriate for prediction for three reasons: (1) the scatter plot shows points clustered tightly around the line; (2) the residual plot for Class 1 shows small, randomly scattered residuals while Class 2 residuals vary widely and unpredictably; (3) Class 1's r of 0.97 is substantially higher than Class 2's r of 0.72, and its r² of 0.94 shows the linear model captures nearly all the variation in scores. Both relationships are linear, but Class 1's is far stronger.

The Class 1 (sleep vs. score) linear model is stronger and more appropriate.

SECTION 7

Interpreting Slope and Intercept in Context

Comparing two linear models also means interpreting what their slopes and intercepts mean in context. Two data sets can both have strong linear relationships but tell very different stories depending on what the slope represents.

Component	What It Means	Example in Context
Slope (b)	The predicted change in y for each one-unit increase in x. A steeper slope means y changes more quickly as x increases.	Class 1: b ≈ 4.7 means each additional hour of sleep is associated with about 4.7 more points on the quiz.
y-intercept (a)	The predicted value of y when x = 0. This may or may not be meaningful depending on whether x = 0 is a realistic value.	Class 1: a ≈ 39.5 would predict a score of 39.5 for zero hours of sleep — plausible as a lower bound, but outside the data range.
Correlation direction	A positive r means y tends to increase as x increases. A negative r means y tends to decrease as x increases.	Both Class 1 and Class 2 have positive r values, meaning more sleep/homework is associated with higher scores.

✦ KEY TAKEAWAY

When comparing two linear models, always interpret both the slope and the correlation in context. A model with a steeper slope is not necessarily better — what matters is how well the linear model fits the data (shown by r and the residual plot) and what the slope and intercept actually mean for the real-world situation being studied.

SECTION 8

Connection to Advanced Topics

The comparison skills you are building now form the foundation for more advanced statistical modeling. In later courses — Math 2, Math 3, AP Statistics, or college-level data analysis — you will encounter tools that formalize and extend these ideas.

What You Learn Now	Where It Leads
Comparing r values for two linear relationships	Comparing linear, quadratic, and exponential models (Math 2/3); adjusted r², AIC, and BIC model selection criteria (AP Statistics, college)
Checking residual plots for patterns vs. random scatter	Formal residual diagnostics: normality tests, heteroscedasticity checks, influence measures
Interpreting slope and intercept in context	Multiple regression with many predictors, polynomial regression, logistic regression for yes/no outcomes
Justifying your model choice with evidence	Hypothesis testing for regression coefficients, confidence intervals for predictions, cross-validation

The key idea carries forward at every level: a model is only as good as the evidence supporting it. Whether you are a Math 1 student comparing two scatter plots or a data scientist choosing among dozens of algorithms, the process is the same — fit, check residuals, compare metrics, and justify.

SECTION 9

Practice Problems

PROBLEM 1 — CONCEPTUAL

A student fits a linear model to a data set and gets r² = 0.89. The residual plot shows a clear U-shaped pattern. Should the student conclude that the linear model is appropriate? Explain why or why not.

PROBLEM 2 — BASIC CALCULATION

Two linear models are fit to two different data sets. For one particular observation, the observed y-value is 45. The linear model for Data Set A predicts ŷ = 50 and the linear model for Data Set B predicts ŷ = 44.5. Calculate the residual for each model at this point and state which model is closer for this observation.

PROBLEM 3 — INTERMEDIATE

Two linear models are each fit to a different data set of 10 points. Model A has r² = 0.85 and its residual plot shows random scatter. Model B has r² = 0.87 and its residual plot also shows random scatter. Which linear relationship is stronger, and by how much? Explain your reasoning.

PROBLEM 4 — APPLIED

A shop owner collects data on daily lemonade sales (y). She fits two separate linear models: one using daily high temperature in °F as x (r = 0.94, r² = 0.88, residuals randomly scattered), and one using dollars spent on advertising as x (r = 0.61, r² = 0.37, residuals randomly scattered). Which linear relationship is stronger? Use all available evidence to justify your answer.

PROBLEM 5 — CRITICAL THINKING

A classmate argues: 'You should always pick the linear model with the highest r² because it explains the most variation.' Write a paragraph explaining why this reasoning is flawed. Use at least two specific scenarios to support your argument.

SUMMARY

Lesson Summary

When comparing two bivariate relationships using linear models, you follow a three-step process. First, examine each scatter plot to confirm the data follows a roughly linear pattern. Second, compare the residual plots of each linear model: a good linear model produces randomly scattered residuals close to zero, while a poor fit shows large scatter or a systematic pattern. Third, compare the correlation coefficient r — a value closer to ±1 means a stronger linear relationship and a more reliable linear model.

Always justify your model choice with multiple pieces of evidence: the shape of the scatter plot, the pattern (or lack thereof) in the residual plot, and the correlation coefficient. Also interpret the slope and intercept in context to explain what the linear relationship actually means for the real-world situation. This skill — evaluating and comparing linear models — is at the heart of data analysis and will serve you in every statistics course you take.