Business Analytics Quiz: Model Diagnostics And Residuals
10 questions · exam conditions
0:00
Model Diagnostics And ResidualsQuestion 1 of 10

A company forecasts quarterly subscription growth. In chronological order, the latest twelve residuals are +4+4, +5+5, +3+3, +6+6, +2+2, +4+4, −3-3, −5-5, −4-4, −6-6, −2-2, and −5-5 percentage points. Their overall average is close to zero.

Which conclusion is most defensible?

The forecasts are independent because positive and negative residuals approximately cancel overall.
The model may omit a time-related pattern because similarly signed residuals occur in long runs.
The model has constant bias because every residual differs materially from zero.
The residuals mainly indicate nonnormality because their signs change halfway through the period.
← Back to quizzes

Business Analytics Quiz

Business Analytics Quiz: Model Diagnostics And Residuals

Practice Model Diagnostics And Residuals in Business Analytics with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Model Diagnostics And Residuals, giving you a quick way to practice the rules, question types, and explanations that matter most for Business Analytics.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

A company forecasts quarterly subscription growth. In chronological order, the latest twelve residuals are +4+4, +5+5, +3+3, +6+6, +2+2, +4+4, −3-3, −5-5, −4-4, −6-6, −2-2, and −5-5 percentage points. Their overall average is close to zero.

Which conclusion is most defensible?

  1. The forecasts are independent because positive and negative residuals approximately cancel overall.
  2. The model may omit a time-related pattern because similarly signed residuals occur in long runs. (correct answer)
  3. The model has constant bias because every residual differs materially from zero.
  4. The residuals mainly indicate nonnormality because their signs change halfway through the period.
Explanation: When evaluating forecast quality, you should always examine residuals as a sequence over time, not just as a collection of numbers. A well-performing model produces residuals that look like random noise — no patterns, no clustering. The moment you spot a systematic structure in residuals, the model is likely missing something important. Here, the first six residuals are all positive (+4,+5,+3,+6,+2,+4+4, +5, +3, +6, +2, +4) and the last six are all negative (−3,−5,−4,−6,−2,−5-3, -5, -4, -6, -2, -5). That long-run clustering of same-signed residuals is a textbook example of autocorrelation — consecutive errors are correlated rather than independent. This strongly suggests the model omits a time-related trend or seasonal shift, making B the most defensible conclusion. A is the classic trap here. Yes, the positives and negatives roughly cancel, producing a near-zero mean — but that only tells you about average bias, not independence. Random errors can cancel on average while still exhibiting a dangerous time pattern. Cancellation ≠ randomness. C is wrong because constant bias would mean residuals consistently sit on one side of zero, not that they differ materially from it. These residuals actually switch signs systematically, which is a pattern problem, not a bias problem. D misidentifies the issue. The sign change halfway through is precisely the evidence of a time pattern, not evidence of nonnormality. Nonnormality concerns the distribution shape of residuals, not their sequential ordering. Study tip: On residual analysis questions, always ask "Do the residuals look random over time?" A near-zero mean is necessary but not sufficient — sequential structure is just as damaging as consistent bias.

Question 2

A loan-loss model is evaluated on equally sized groups of applicants from two sales channels. Its overall mean residual is zero. However, the online channel has a mean residual of +8+8 dollars, while the branch channel has a mean residual of −8-8 dollars. Residuals are defined as actual loss minus predicted loss.

What does this diagnostic result imply?

  1. The model is unbiased for both channels because the combined residual mean equals zero.
  2. The model overpredicts online losses and underpredicts branch losses by offsetting amounts.
  3. The model underpredicts online losses and overpredicts branch losses by offsetting amounts. (correct answer)
  4. The model has only a variance problem because channel-level residual means have opposite signs.
Explanation: When evaluating a predictive model, you need to understand what the sign of a residual tells you. Since residuals are defined as actual minus predicted, a positive mean residual means actual losses exceeded predictions — the model underpredicted. A negative mean residual means predictions exceeded actual losses — the model overpredicted. Applying this to the passage: the online channel has a mean residual of +8+8, meaning actual losses were higher than predicted — the model underpredicted online losses. The branch channel has a mean residual of −8-8, meaning predictions were higher than actual losses — the model overpredicted branch losses. These two errors cancel out arithmetically, producing an overall mean of zero. That makes C the correct answer. A is wrong because a combined residual of zero does not mean the model is unbiased for each group — it can simply mean two opposite biases are masking each other. This is a classic Simpson's Paradox-adjacent trap: aggregate statistics can hide subgroup problems. B reverses the logic entirely. It claims the model overpredicts online and underpredicts branch losses — the opposite of what +8+8 and −8-8 actually indicate given the residual definition. D misdiagnoses the problem type. Opposite signs in subgroup means indicate a bias (systematic directional error), not a variance problem. Variance issues relate to the spread or inconsistency of residuals, not their average direction. As a study habit, always anchor on the residual definition given in the question — exam writers frequently flip the formula to test whether you read carefully before interpreting sign direction.

Question 3

In a regression predicting store profit from floor space, one newly opened flagship store has floor space far beyond that of every other store. Its residual is only 11 thousand dollars, compared with a typical absolute residual of 88 thousand dollars. When this store is removed, the estimated floor-space coefficient changes substantially.

How should the flagship store be characterized?

  1. It is influential because its unusual predictor value materially affects the fitted coefficient. (correct answer)
  2. It is a response outlier because its residual is much larger than typical residuals.
  3. It is harmless because an observation with a small residual cannot affect regression estimates.
  4. It demonstrates heteroscedasticity because one extreme predictor value has a small residual.
Explanation: Whenever regression questions describe an unusual predictor value, you need to think carefully about two distinct concepts: outliers (unusual in the response) and influential observations (unusual in the predictor, capable of pulling the regression line). These concepts are easy to confuse because students often assume a "problem" observation must have a large residual. An observation is influential when it has high leverage — meaning its predictor value sits far from the rest of the data — AND its removal materially changes the estimated coefficients. The flagship store has floor space far beyond every other store, giving it extreme leverage. Even though its residual is small (11 thousand vs. the typical 88 thousand), removing it substantially shifts the floor-space coefficient. That's the textbook definition of an influential point. Answer A correctly captures this: the unusual predictor value is what drives the influence, not the size of the residual. Answer B is wrong because a response outlier is defined by having an unusually large residual — the flagship store's residual is actually smaller than typical, so this label is backwards. Answer C contains a dangerous half-truth: a small residual does not protect an observation from being influential. In fact, high-leverage points often have small residuals precisely because the regression line bends toward them, which is exactly what makes them so impactful and hard to detect visually. Answer D misapplies heteroscedasticity, which refers to non-constant variance of residuals across the range of predictors — a single data point's residual pattern says nothing about that structural property. Study tip: Remember that high leverage + coefficient instability = influential observation, regardless of residual size. Always check both dimensions when evaluating unusual data points.

Question 4

A purchase model assigns each customer a probability. Among 200200 customers assigned probabilities near 0.700.70, exactly 120120 make a purchase. For diagnostic purposes, each customer's residual is defined as observed outcome minus predicted probability, where a purchase is 11 and no purchase is 00.

What is the approximate mean residual for this probability group, and how should it be interpreted?

  1. The mean is +0.10+0.10, so the model underpredicts purchases by about ten percentage points for this group.
  2. The mean is −0.70-0.70, so the model overpredicts purchases by about seventy percentage points for this group.
  3. The mean is +0.60+0.60, so the model underpredicts purchases by about sixty percentage points for this group.
  4. The mean is −0.10-0.10, so the model overpredicts purchases by about ten percentage points for this group. (correct answer)
Explanation: When a model assigns probabilities to customers, you can evaluate its calibration by computing residuals — the difference between what actually happened and what the model predicted. For a binary outcome, each residual is simply observed−predicted\text{observed} - \text{predicted}, where observed is either 1 (purchased) or 0 (did not purchase). Here's the calculation: out of 200 customers assigned a probability near 0.70, exactly 120 made a purchase. The observed purchase rate is 120/200=0.60120/200 = 0.60. The mean residual equals the average observed outcome minus the predicted probability: 0.60−0.70=−0.100.60 - 0.70 = -0.10. This negative value tells you the model is overpredicting — it assigned 70% probability to a group that only purchased 60% of the time. That makes D correct. Looking at the distractors: A gets the magnitude right (10 percentage points) but flips the sign and the interpretation. A mean of +0.10+0.10 would mean actual purchases exceeded the model's prediction, indicating underprediction — the opposite of what happened here. B confuses the predicted probability itself (0.70) with the residual, as if the model predicted perfectly but you're subtracting the full probability from zero; this conflates the prediction with the error. C mistakes the observed rate (0.60) for the residual, forgetting that residuals require subtracting the predicted value from the observed value, not reporting the observed rate alone. A useful rule of thumb: negative mean residual = overprediction, positive mean residual = underprediction. When the model's predicted probability exceeds the group's actual purchase rate, the residuals are systematically negative — the model was too optimistic about that segment.

Question 5

An analyst groups a demand model's observations from lowest to highest fitted demand. The mean residuals in the five groups are, respectively, +12+12, −6-6, −11-11, −5-5, and +10+10 units. The overall mean residual is approximately zero.

What is the most appropriate diagnostic conclusion?

  1. The intercept should be increased because the residuals are positive at both extremes.
  2. The error variance is nonconstant because the residual means differ across groups.
  3. The model likely misses a nonlinear relationship because residual means follow a curved pattern. (correct answer)
  4. The model is adequately specified because its overall mean residual is approximately zero.
Explanation: Whenever you see a question involving residuals grouped by fitted values, you should be thinking about model misspecification diagnostics — specifically, whether the pattern of residuals reveals a systematic flaw in how the model is structured. Look at the residual means across the five groups: +12,−6,−11,−5,+10+12, -6, -11, -5, +10. Plot these mentally against fitted demand from low to high. The pattern starts high, dips down through the middle, then rises again — a classic U-shaped (or curved) pattern. This tells you the model is systematically over- or under-predicting at different ranges of fitted values in a structured way, which is the hallmark of a missed nonlinear relationship. A linear model applied to inherently nonlinear data will always leave behind this kind of curved residual signature. Answer C is correct. Answer A is tempting because the residuals are indeed positive at both extremes, but the conclusion — adjust the intercept — is wrong. Shifting the intercept up or down moves all predictions uniformly; it cannot fix a curved pattern that flips signs in the middle. A is diagnosing the wrong remedy entirely. Answer B points to heteroscedasticity (nonconstant error variance), but that would involve residual spread (variance) changing across groups, not residual means following a curve. The pattern here is about systematic bias, not changing variability. Answer D is a classic trap: a near-zero overall mean residual only means errors roughly cancel out in aggregate. It says nothing about whether errors are randomly distributed — systematic patterns within subgroups can still indicate serious misspecification. Study tip: Always look at residual patterns, not just summary statistics. A zero mean can hide a serious curved pattern that screams "add a quadratic term."

Question 6

A revenue model's training residuals show no meaningful pattern across fitted-value groups. On a later validation period, low predictions have positive mean residuals, middle predictions have mean residuals near zero, and high predictions have negative mean residuals. Residuals are actual revenue minus predicted revenue.

Which interpretation is best supported by the validation diagnostics?

  1. The validation predictions are too extreme, suggesting poor generalization or a need for recalibration. (correct answer)
  2. The model needs only a larger intercept because all validation predictions are too low.
  3. The model generalizes well because positive and negative validation residuals can cancel overall.
  4. The model has only nonconstant variance because residual means change with predicted revenue.
Explanation: When evaluating model diagnostics, you need to distinguish between training performance and validation performance — and pay close attention to the pattern of residuals, not just their average. Here, residuals are defined as actual minus predicted, so a positive residual means underprediction and a negative residual means overprediction. The key signal is the fan-shaped pattern on validation: low predictions have positive residuals (the model underpredicts low-revenue cases) and high predictions have negative residuals (the model overpredicts high-revenue cases). This classic pattern means the model's predictions are too spread out — the lows are too low and the highs are too high. The model is overconfident in its extremes, a hallmark of poor generalization or a model that needs recalibration toward the center. Since training residuals showed no such pattern, the problem emerged specifically on new data, confirming generalization failure. Answer A is correct. Answer B is wrong because not all predictions are too low — high predictions are actually too high (negative residuals), so a blanket intercept adjustment would help one group while hurting the other. Answer C is wrong because cancellation of positive and negative residuals is misleading. A near-zero overall mean can mask a serious systematic pattern. Good calibration requires residuals to be near zero within groups, not just on average. Answer D is wrong because nonconstant variance (heteroscedasticity) refers to the spread of residuals changing across groups, not the mean. What's described here is a bias pattern — shifting means — not a variance issue. Study tip: When residual means shift systematically with predicted values, always think bias pattern, not just variance. Cancellation across groups is one of the most common traps in model diagnostics questions.

Question 7

A retailer defines a forecast residual as actual weekly sales minus predicted weekly sales. For one store, the model predicted sales of 480480 units, while actual sales were 520520 units.

Which interpretation of this observation is correct?

  1. The residual is −40-40 units, indicating that the model overpredicted sales.
  2. The residual is +40+40 units, indicating that the model underpredicted sales. (correct answer)
  3. The residual is +40+40 units, indicating that the model overpredicted sales.
  4. The residual is −40-40 units, indicating that the model underpredicted sales.
Explanation: When working with residuals in forecasting, always anchor yourself to the definition: residual = actual − predicted. The sign and magnitude of that difference tell you exactly how the model performed relative to reality. Here, the calculation is straightforward: 520−480=+40520 - 480 = +40 units. The positive sign means actual sales exceeded what the model expected — the model fell short of reality, which is called underprediction. That makes B correct: a residual of +40+40 units, indicating the model underpredicted sales. A gets the sign wrong (−40-40) and then compounds the error by calling it an overprediction. A negative residual would mean actual < predicted — the model guessed too high — so both the number and the label are flipped. C uses the correct residual of +40+40 but misidentifies what it means: a positive residual cannot indicate overprediction, because the model would need to have predicted more than what actually sold for that to be true. D applies the wrong sign (−40-40) but accidentally pairs it with the correct interpretation label ("underpredicted"), creating a mismatched combination that looks plausible but is numerically wrong. A reliable memory trick: think of the residual's sign from the model's perspective. A positive residual means the model was caught under — reality came in higher than it expected. A negative residual means the model overshot. On exam questions, watch for distractors that correctly calculate the residual but swap the directional language, or get the sign wrong while using the right label.

Question 8

Two models are evaluated on the same five validation cases. Model A has residuals −1-1, −1-1, −1-1, −1-1, and +8+8. Model B has residuals −3-3, −3-3, −3-3, −3-3, and +3+3. The business considers occasional very large errors especially costly.

Which comparison best supports a model choice?

  1. Choose Model A because its mean absolute error is lower, which is the appropriate metric regardless of error shape.
  2. Choose Model B because its mean absolute error is lower, which already accounts for the cost of large errors.
  3. Choose Model A because its residual mean is closer to zero, indicating less systematic bias overall.
  4. Choose Model B because its root mean squared error is lower and large errors are penalized heavily by that metric. (correct answer)
Explanation: When a business explicitly flags large errors as especially costly, that's your cue to focus on which metric penalizes large errors most heavily — and that metric is Root Mean Squared Error (RMSE), because squaring residuals amplifies the impact of outliers disproportionately. Let's run the numbers. For Model A, the squared residuals are 1,1,1,1,641, 1, 1, 1, 64, giving MSE =68/5=13.6= 68/5 = 13.6 and RMSE≈3.69\text{RMSE} \approx 3.69. For Model B, the squared residuals are 9,9,9,9,99, 9, 9, 9, 9, giving MSE =45/5=9.0= 45/5 = 9.0 and RMSE=3.0\text{RMSE} = 3.0. Model B wins clearly under RMSE because Model A's single +8+8 residual balloons its squared-error sum. Since the business cares about those large spikes, RMSE is the right lens — making D correct. A is wrong because MAE treats all errors equally (no squaring), so it doesn't reflect the extra cost of large errors. Model A's MAE is 1+1+1+1+85=2.4\frac{1+1+1+1+8}{5} = 2.4 versus Model B's 3+3+3+3+35=3.0\frac{3+3+3+3+3}{5} = 3.0, so A wins on MAE — but MAE is the wrong metric for this cost structure, making choice A's reasoning doubly flawed. B is wrong for the same reason: MAE is cited as the justification, but MAE does not account for the extra cost of large errors. The conclusion happens to favor the right model, but for the wrong reason. C is wrong because both models have a residual mean of exactly zero (−4+8=4−4=0-4 + 8 = 4 - 4 = 0), so this criterion doesn't differentiate them at all. Study tip: On metric-selection questions, match the metric to the cost structure. Large errors especially costly → RMSE. Errors treated equally → MAE. Don't let a lower MAE distract you when the problem context demands a squared-error metric.

Question 9

For a model predicting customer account value, residuals are tightly clustered for low fitted values but become increasingly dispersed as fitted values rise. Within each fitted-value range, the residuals remain centered near zero.

Which action and interpretation are best supported by this pattern?

  1. Increase the intercept because the model systematically underpredicts high-value accounts.
  2. Investigate nonconstant error variance because equal-width prediction intervals may be misleading. (correct answer)
  3. Add a nonlinear mean term because increasing residual spread proves that the fitted mean is curved.
  4. Remove high-value accounts because their larger residuals show that they are invalid observations.
Explanation: Whenever you see residuals that behave differently across the range of fitted values, your first instinct should be to diagnose heteroscedasticity — nonconstant error variance. The key diagnostic here is the spread of residuals, not their center. The passage tells you residuals are centered near zero throughout, meaning the model's mean predictions are unbiased. But the fan-shaped spread (tight at low values, wide at high values) is the textbook signature of heteroscedasticity. This confirms B as the correct action. When error variance grows with fitted values, ordinary prediction intervals — which assume constant variance — will be too narrow for high-value accounts and too wide for low-value ones. Flagging and investigating this pattern (often via a scale-location plot or Breusch-Pagan test) is exactly the right response. A is wrong because systematic underprediction would appear as residuals consistently above zero for high fitted values, not just more dispersed around zero. The intercept adjustment addresses mean bias, not variance spread. C is wrong because nonlinearity would show up as residuals that are systematically positive or negative in certain ranges — a curved pattern in the mean. Increasing spread alone says nothing about whether the fitted mean is curved; these are separate diagnostic concerns. D is wrong because larger residuals in a group don't automatically make observations invalid. Heteroscedasticity is a structural feature of the data-generating process, not a data quality problem. Removing high-value accounts would introduce bias and waste information. Study tip: Always separate two residual diagnostics — where they're centered (mean bias, nonlinearity) versus how spread out they are (variance structure). Fan patterns = variance problem, not mean problem.

Question 10

A delivery-time model has a residual root mean squared error of 2020 minutes. For one delivery, the predicted time is 110110 minutes and the actual time is 6060 minutes. For a preliminary diagnostic, the analyst scales the residual by the model's root mean squared error.

What is the scaled residual, and what does it indicate?

  1. It is −2.5-2.5, indicating an unusually large overprediction for that delivery. (correct answer)
  2. It is +2.5+2.5, indicating an unusually large underprediction for that delivery.
  3. It is −0.4-0.4, indicating a relatively ordinary overprediction for that delivery.
  4. It is +0.4+0.4, indicating a relatively ordinary underprediction for that delivery.
Explanation: When diagnosing model performance, analysts often scale residuals by dividing them by the model's root mean squared error (RMSE). This converts a raw residual into a standardized unit, similar in spirit to a z-score, letting you judge whether a prediction error is unusually large or relatively routine. A scaled residual beyond roughly ±2 is generally flagged as noteworthy. The raw residual is always computed as actual minus predicted: 60−110=−5060 - 110 = -50 minutes. The negative sign tells you the model overpredicted — it guessed the delivery would take longer than it actually did. Scaling that by the RMSE gives −5020=−2.5\frac{-50}{20} = -2.5. Because the magnitude exceeds 2, this residual is unusually large, confirming a substantial overprediction. That's exactly what A describes, making it the correct answer. B is wrong on two counts: the sign should be negative (not positive), and calling it an underprediction reverses the direction — an underprediction would mean actual > predicted. C gets the sign right and correctly identifies it as an overprediction, but the arithmetic is inverted: 2050=0.4\frac{20}{50} = 0.4 instead of −5020=−2.5\frac{-50}{20} = -2.5. That's the trap of dividing RMSE by the residual rather than the other way around. D compounds C's arithmetic error with the wrong direction label. Study tip: Always anchor residuals with the formula actual − predicted before scaling. A negative residual = overprediction (model shot too high); a positive residual = underprediction (model shot too low). Getting the sign right is half the battle on these questions.