Residuals & Diagnostics - Business Statistics

1. Given the inverted U-shaped pattern in the residual plot, which of the following is the most appropriate next step for the analyst to improve the model?
   Answer: Add a quadratic term (speed$^2$) for the production line speed to the regression model. An inverted U-shaped pattern in the residuals indicates that the relationship between the predictor (speed) and the response (defects) is quadratic, not linear. The standard and most direct way to model this type of curvature is to add a polynomial term, specifically a quadratic term (speed$^2$), to the model. The new model would be $\text{defects} = \beta_0 + \beta_1 \times \text{speed} + \beta_2 \times \text{speed}^2$. A addresses heteroscedasticity, which is not the primary issue described. B is a transformation often used for different types of non-linearity or to stabilize variance. D is inappropriate because removing data that reveals a systematic pattern is poor practice.
2. A marketing analyst runs a regression to predict product sales from advertising spend. A single data point, representing a recent promotional campaign, is found to have both very high leverage and a large standardized residual. What is the most professionally sound course of action?
   Answer: The analyst should investigate the context of the promotional campaign to understand why it was unusual before deciding on its treatment. A point with both high leverage and a large residual is an influential point. Such points have a large impact on the regression results. The most prudent and professional action is not to automatically remove or keep the point, but to investigate it. Understanding the context is key. Was there a data entry error? Was this campaign fundamentally different from others (e.g., run during a holiday, combined with a price cut)? This information is crucial for deciding whether the point belongs in the model, whether the model needs to be modified (e.g., by adding another variable), or if the point should be excluded. A and B are too dogmatic, and D suggests a remedy without a proper diagnosis.
3. In a regression analysis, observation #47 has a hat value (leverage) of 0.15, DFBETAS value of 0.8 for the slope coefficient, and DFFITS value of 0.65. For a dataset with n=60 and k=3 predictors, the suggested cutoffs are: hat value > 2k/n, |DFBETAS| > 2/√n, and |DFFITS| > 2√(k/n). What is the most accurate assessment of this observation's influence?
   Answer: High influence on both coefficient estimates and fitted values; immediate investigation and potential removal is warranted Critical values: hat > 2(3)/60 = 0.10; |DFBETAS| > 2/√60 = 0.258; |DFFITS| > 2√(3/60) = 0.447. The observation substantially exceeds all thresholds: leverage (0.15 vs 0.10), DFBETAS (0.8 vs 0.258), and DFFITS (0.65 vs 0.447). This indicates high influence on both coefficient estimates and fitted values, warranting immediate investigation.
4. In a regression analysis of 200 data points, one observation has a studentized residual of -3.15. Which of the following is the most accurate conclusion?
   Answer: This observation has a value for the dependent variable that is much lower than predicted by the model and is a likely outlier. Studentized (or standardized) residuals measure how many standard deviations an observed value is from its predicted value. A common rule of thumb is that absolute values greater than 2 are of concern, and values greater than 3 are very likely outliers. A value of -3.15 indicates the actual y-value is more than 3 standard deviations below the predicted value, making it a likely outlier. Distractor A is too strong; a point can be an outlier without being influential, and removal is not automatic. Distractor C confuses a large residual (outlier) with high leverage (extremeness in x-values). Distractor D is an overstatement; a single outlier does not necessarily invalidate the overall normality assumption for the other 199 residuals.
5. A business analyst fits an ordinary least squares (OLS) regression model to a dataset that exhibits significant heteroscedasticity. If no corrective measures are taken, such as using robust standard errors or applying a transformation, what is the most critical consequence for the model's results?
   Answer: The F-test for the overall significance of the model may become unreliable. In the presence of heteroscedasticity, OLS coefficient estimates remain unbiased and consistent. However, the formulas for their standard errors are incorrect (usually underestimated). Since the t-tests for individual coefficients and the F-test for overall model significance both depend on these standard errors, they become unreliable. This can lead to incorrect conclusions about the statistical significance of the predictors. A is incorrect because the coefficient estimates are not biased. C is incorrect as R-squared is not systematically biased in one direction. D is incorrect because heteroscedasticity and non-normality are separate issues.
6. A regression model is built to forecast monthly sales using data from the past 60 months. The Durbin-Watson statistic for the model's residuals is calculated to be 0.78. What does this value suggest about the model?
   Answer: The model's residuals exhibit significant positive autocorrelation. The Durbin-Watson (DW) statistic tests for first-order autocorrelation in the residuals. The statistic ranges from 0 to 4. A value of 2.0 indicates no autocorrelation. Values approaching 0 indicate positive autocorrelation, while values approaching 4 indicate negative autocorrelation. A value of 0.78 is significantly less than 2.0, providing strong evidence of positive autocorrelation. This is a common issue in time-series data where a positive error in one period is likely to be followed by another positive error. Distractor B is incorrect; the DW value would be greater than 2. Distractor C incorrectly states the expected value is 1.0. Distractor D confuses autocorrelation with heteroscedasticity.
7. After running a multiple regression, a data scientist notes the following: 1) The VIF values for all predictors are below 2. 2) The Durbin-Watson statistic is 1.95. 3) A plot of residuals versus predicted values shows a random scatter of points in a horizontal band. 4) The normal Q-Q plot of the residuals shows the points hugging the diagonal line closely, except for two points at the extreme upper end that are far above the line. Which conclusion is most justified?
   Answer: The key model assumptions appear to be met, but there may be a few outliers that warrant further investigation. When evaluating multiple regression models, you need to systematically check key assumptions: multicollinearity, autocorrelation, homoscedasticity, and normality of residuals. This question tests your ability to interpret diagnostic statistics and plots together. The correct answer is D because each diagnostic indicates the model is fundamentally sound. VIF values below 2 show no multicollinearity (values above 5-10 indicate problems). The Durbin-Watson statistic of 1.95 is very close to 2, indicating no autocorrelation. The random scatter in the residuals vs. predicted plot confirms homoscedasticity (constant variance). The Q-Q plot shows normality overall, with just two extreme points deviating—classic signs of potential outliers rather than systematic assumption violations. Option A is wrong because VIF values below 2 actually indicate absence of multicollinearity, not its presence. Option B misinterprets the evidence—a non-linear relationship would show patterns in the residuals vs. predicted plot, but you see random scatter, which is exactly what you want. Option C confuses concepts: heteroscedasticity refers to non-constant variance in residuals (shown in residuals vs. predicted plots), not outliers in normality plots. The two extreme points in the Q-Q plot suggest outliers, not variance issues. Remember this diagnostic sequence: VIF for multicollinearity, Durbin-Watson for autocorrelation, residual plots for linearity/homoscedasticity, and Q-Q plots for normality. When most diagnostics look good but you see a few extreme points, think outliers rather than assumption violations.
8. In the context of regression diagnostics, the leverage of an individual observation is primarily determined by which of the following?
   Answer: Its distance from the center (mean) of the values of the independent variable(s), $x$. Leverage is a measure of how much an observation's values for the independent variable(s) (x-values) deviate from the average of the x-values in the dataset. Points with extreme x-values are high-leverage points and have more potential to influence the regression line. A describes an outlier. B is incorrect. D is incorrect because leverage is an input into the calculation of Cook's distance, not the other way around. Cook's distance measures influence, which is a combination of leverage and outlier status.
9. An analyst models a company's quarterly profit based on its marketing expenditure. A plot of the residuals versus the fitted (predicted) profit values shows a distinct U-shape. What is the most likely issue with the model and what is the most appropriate next step?
   Answer: The relationship between the variables is non-linear, and the analyst should consider adding a squared term for marketing expenditure to the model. A U-shaped pattern in a plot of residuals versus fitted values indicates that the linear model is not capturing a non-linear relationship. The most common way to model such a quadratic relationship is to add a squared term (e.g., marketing expenditure squared) as a new predictor in the model. Distractor A describes the remedy for a different issue (heteroscedasticity). Distractor C misdiagnoses the problem and suggests a potentially inappropriate action. Distractor D describes autocorrelation, which is typically identified by a Durbin-Watson test or a plot of residuals over time, not a U-shape in a residuals vs. fitted plot.
10. A regression model predicting sales has studentized residuals where 3 out of 100 observations exceed ±2.5 in absolute value, and 1 observation has a studentized residual of -3.2. The Q-Q plot of residuals shows points generally following the diagonal line but with some curvature in the tails. What is the most reasonable assessment?
    Answer: Moderate departure from normality with potential outliers; investigate the extreme observations but inference remains reasonably robust When evaluating regression model assumptions, you need to assess residual patterns systematically rather than jumping to conclusions from isolated statistics. This question tests your ability to interpret multiple diagnostic indicators together. The evidence here suggests moderate departure from normality with some outliers, making D correct. Having 3% of observations exceed ±2.5 in studentized residuals is slightly elevated (you'd expect about 1-2% under perfect normality), and the -3.2 value indicates a potential outlier. The Q-Q plot showing general adherence to the diagonal with tail curvature is classic for mild normality violations. Importantly, regression inference is robust to moderate departures from normality, especially with reasonable sample sizes like n=100. Option A is too extreme—these patterns don't indicate "serious" violations requiring non-parametric methods. The deviations are manageable and common in real data. Option B misinterprets the evidence entirely; nothing here suggests heteroscedasticity (non-constant variance). Studentized residuals exceeding thresholds and Q-Q plot curvature relate to normality, not variance patterns. Option C is too dismissive—while the violations aren't severe, calling the normality assumption "satisfied" ignores clear evidence of departure. The key insight is that real data rarely perfectly follows theoretical distributions, but statistical methods remain valid under moderate violations. The pattern described falls into this "investigate but proceed" category rather than "panic and abandon the model" or "ignore completely." Study tip: Learn to distinguish between minor assumption violations (which are common and manageable) versus serious violations requiring model changes. Focus on the magnitude and practical impact, not perfect adherence to theoretical ideals.
11. A residual analysis reveals that 95% of standardized residuals fall within ±1.8, 99% fall within ±2.4, and the largest absolute residual is 2.9. The histogram of residuals appears roughly symmetric but slightly more peaked than a normal distribution. What diagnostic conclusion is most appropriate?
    Answer: Residuals demonstrate approximate normality with slight excess kurtosis; the departure is minor and inference remains valid The percentile values (95% within ±1.8 vs. normal expectation of ±1.96, 99% within ±2.4 vs. ±2.58) and maximum residual of 2.9 suggest slight leptokurtosis (more peaked) but close to normal. This minor departure doesn't invalidate inference. Choice A overstates the issue, Choice B misidentifies the distribution type, and Choice D incorrectly suggests heavy-tail problems when the distribution is actually more concentrated.
12. A real estate analyst models the price of apartments based on their size in square feet. A diagnostic check reveals heteroscedasticity, where the variance of the residuals increases as the predicted price increases. What is the most likely practical interpretation of this finding?
    Answer: The model's absolute prediction error (in dollars) tends to be larger for more expensive apartments. Heteroscedasticity in this context means the spread of the errors (residuals) is not constant. If the variance of the residuals increases with the predicted price, it means that for higher-priced apartments, the data points are more scattered around the regression line. This translates to larger typical prediction errors (in absolute dollar amounts) for expensive apartments compared to inexpensive ones. Distractor A describes homoscedasticity. Distractor B has the relationship reversed. Distractor D confuses heteroscedasticity with a violation of the linearity assumption.
13. A time-series regression shows residuals where $e_t = 0.6e_{t-1} + \epsilon_t$, where $\epsilon_t$ is white noise. If the original model had $R^2 = 0.75$ and the Durbin-Watson statistic is 0.95, what adjustment would most effectively address the primary concern?
    Answer: Add lagged values of the dependent variable as explanatory variables to capture the autoregressive structure The residual pattern $e_t = 0.6e_{t-1} + \epsilon_t$ indicates first-order autocorrelation (confirmed by DW = 0.95 < 2). Adding lagged dependent variables captures this autoregressive structure. Choice A addresses heteroscedasticity, Choice C doesn't specifically target autocorrelation, and Choice D is for heteroscedasticity, not autocorrelation.
14. A regression model with 80 observations yields residuals where the first 40 observations (time period 1) have mean residual = 0.15 and standard deviation = 2.1, while the last 40 observations (time period 2) have mean residual = -0.15 and standard deviation = 3.8. A plot of residuals versus observation order shows this distinct pattern. What is the primary concern and most appropriate remedy?
    Answer: Structural break in the relationship; test for parameter stability and consider separate models for each time period The systematic shift in mean residuals between periods (0.15 to -0.15) combined with changing variance suggests a structural break - the underlying relationship changed between periods. This requires testing parameter stability and potentially fitting separate models. Choice B addresses correlation but not the mean shift, Choice C only addresses variance changes, and Choice D assumes measurement error without evidence.
15. Based on the information provided, how would the data point for the specialty health food shop ($5.00, 5 thousand units) most likely be characterized in the regression diagnostics?
    Answer: As a high-leverage point with a large residual, making it a likely influential point. This is a two-step problem. First, assess leverage. Leverage is determined by the distance of the x-value from the mean of x. Here, the price of $5.00 is far from the mean price of $2.50, so the point has high leverage. Second, assess the residual. The predicted sales at a price of $5.00 are $\hat{y} = 150 - 40(5) = 150 - 200 = -50$ thousand units. The actual sales were 5 thousand units. The residual is $e = y - \hat{y} = 5 - (-50) = 55$. This is a very large residual. A point with both high leverage and a large residual is very likely to be an influential point, meaning it has a large effect on the estimated regression line. Therefore, C is the best characterization.
16. A consultant builds a linear regression model to predict customer lifetime value and reports an $R^2$ of 0.92. However, a plot of the model's residuals against the predicted values reveals a distinct curved (parabolic) pattern. What is the most appropriate conclusion?
    Answer: The model is fundamentally flawed because the linearity assumption is violated, and the high $R^2$ is misleading. When evaluating regression models, you must examine both goodness-of-fit statistics like $R^2$ and diagnostic plots to assess model validity. A high $R^2$ only tells you how much variance the model explains—it doesn't guarantee the model assumptions are met. The curved pattern in the residual plot is a red flag indicating the linearity assumption is violated. Linear regression assumes the relationship between variables is linear, but the parabolic residuals suggest a non-linear relationship that the straight-line model cannot properly capture. When this fundamental assumption fails, the model's predictions become unreliable despite the impressive $R^2$ value. The high $R^2$ is misleading because it reflects the model's ability to fit the data points, but fitting data incorrectly doesn't create a valid model. Option A is wrong because residual patterns should never be ignored—they reveal critical assumption violations that invalidate the model regardless of $R^2$. Option B incorrectly attributes the curved pattern to outliers, but systematic curvature indicates a structural model problem, not random outliers. Option C confuses the issue—multicollinearity affects relationships between predictor variables and typically doesn't create curved residual patterns; it's diagnosed through different methods like variance inflation factors. Remember this key principle: always examine residual plots before trusting any regression model, no matter how high the $R^2$. Curved residuals typically signal you need a non-linear model, polynomial terms, or variable transformations. Good model diagnostics prevent costly business decisions based on fundamentally flawed predictions.
17. An analyst is examining the results of a multiple regression model with 150 observations and 4 predictor variables. The analyst calculates Cook's distance for each observation and finds one observation with a Cook's distance of 1.15. What is the most appropriate interpretation and subsequent action?
    Answer: The value is high, indicating the point is highly influential; the analyst should investigate the point for potential data entry errors or unique circumstances. A common rule of thumb is that a Cook's distance value greater than 1 indicates a highly influential observation. A value of 1.15 is therefore a strong warning sign. The most professionally sound action is not to remove the point automatically but to investigate it. It could be a data entry error, or it could represent a genuinely unique case that needs to be understood before deciding on its inclusion or exclusion. Distractor A uses an incorrect threshold (4/n is a much lower bar for flagging points for inspection). Distractor B suggests immediate removal, which is too hasty. Distractor D incorrectly describes Cook's distance; it specifically measures influence on the regression coefficients.
18. A financial analyst models the daily return of a specific stock (Y) based on the daily return of the S&P 500 index (X). The fitted regression line is $\hat{Y} = 0.0005 + 1.2X$. On a particular day, the S&P 500 index had a return of +0.015 (1.5%), and the actual stock return was +0.021 (2.1%). What is the residual for this data point?
    Answer: 0.0025 The residual is the difference between the actual observed value (y) and the predicted value ($\hat{y}$). First, calculate the predicted value: $\hat{y} = 0.0005 + 1.2 \times 0.015 = 0.0005 + 0.018 = 0.0185$. Next, calculate the residual: $e = y - \hat{y} = 0.021 - 0.0185 = 0.0025$. Distractor B has the wrong sign ($\hat{y} - y$). Distractor C is the predicted value ($\hat{y}$), not the residual. Distractor D is an incorrect calculation of the predicted value.
19. In a multiple regression with 50 observations, one data point has a leverage value of 0.18, a standardized residual of -2.8, and a Cook's distance of 0.35. Given that the critical values are: leverage > 0.12, |standardized residual| > 2.5, and Cook's distance > 0.25, what is the most appropriate classification and recommended action?
    Answer: High-influence outlier; investigate the observation and consider removing it as it likely distorts the regression results significantly This point exceeds all three critical thresholds, making it a high-influence outlier that affects both the fitted line's position and slope. With high leverage (unusual X), large residual (doesn't fit the pattern), and high Cook's distance (changes regression significantly), investigation and potential removal is warranted. The other choices underestimate the combined impact of meeting all three criteria.
20. A regression model is used to predict employee salaries based on years of experience. A diagnostic plot of the model's residuals versus the predicted salaries reveals a funnel shape, with the spread of the residuals increasing as the predicted salary increases. What is the primary consequence of this observed pattern for the OLS regression?
    Answer: The standard errors for the model's coefficients are unreliable, making hypothesis tests and confidence intervals for the coefficients invalid. The funnel shape in the residual plot is a classic sign of heteroscedasticity (non-constant variance of errors). While heteroscedasticity does not cause bias in the OLS coefficient estimates themselves, it does lead to incorrect (typically underestimated) standard errors. This invalidates any statistical inference based on those standard errors, such as t-tests for coefficient significance and confidence intervals. A describes bias, which is incorrect. B is incorrect as R-squared is not systematically affected in this way. C is incorrect as this pattern relates to variance, not the linearity assumption.

Start this quiz