Multiple Regression - Business Statistics

A researcher wants to determine if adding two new variables, $X_3$ and $X_4$ , improves a regression model that already contains $X_1$ and $X_2$ . The "reduced" model (with $X_1, X_2$ ) has an R-squared of 0.40, while the "full" model (with $X_1, X_2, X_3, X_4$ ) has an R-squared of 0.45. A partial F-test is conducted to test the joint significance of $X_3$ and $X_4$ ( $H_0: \beta_3 = \beta_4 = 0$ ). The p-value for this F-test is found to be 0.03. At a 5% significance level, what is the correct interpretation?
Answer: The null hypothesis is rejected, indicating that the full model provides a statistically significant improvement in fit over the reduced model. When you encounter questions about comparing nested regression models, you're dealing with partial F-tests that assess whether adding variables significantly improves model performance beyond what you'd expect by chance. The partial F-test here tests $H_0: \beta_3 = \beta_4 = 0$ (the new variables have no effect) against $H_1$ : at least one coefficient is non-zero. With a p-value of 0.03, which is less than the 0.05 significance level, you reject the null hypothesis. This means the data provides strong evidence that adding $X_3$ and $X_4$ creates a statistically significant improvement over the reduced model. Answer A correctly captures this interpretation. Answer B makes the common mistake of confusing statistical significance with practical significance. While the R-squared increase of 0.05 might seem small, statistical significance is determined by the F-test p-value, not the magnitude of R-squared change. Answer C gets the hypothesis testing backwards—when p < α, you reject (not accept) the null hypothesis. Answer D confuses joint significance with individual significance. A significant partial F-test only tells you that at least one variable contributes significantly as a group; it doesn't guarantee that each variable would be individually significant in separate t-tests. Remember that partial F-tests evaluate the collective contribution of multiple variables simultaneously. Focus on comparing the p-value to your significance level—if p < α, the additional variables provide statistically significant improvement, regardless of how small the R-squared increase appears.
A real estate analyst models house prices using square footage (SQFT, in square feet) and age of the house (AGE, in years). To account for how the market values square footage differently for newer versus older homes, an interaction term is included. The fitted regression equation is: $\text{Price} = 150,000 + 120(\text{SQFT}) - 500(\text{AGE}) - 2(\text{SQFT} \times \text{AGE})$ Based on this model, how should the effect of an additional square foot of space be interpreted for a 10-year-old house?
Answer: The estimated price increases by $100. To find the effect of an additional square foot, we need to consider both the main effect of SQFT and its interaction with AGE. The partial derivative of Price with respect to SQFT is $120 - 2(\text{AGE})$ . For a 10-year-old house, AGE = 10. Plugging this into the expression gives the marginal effect of SQFT as $120 - 2(10) = 120 - 20 = 100$ . Therefore, for a 10-year-old house, each additional square foot is associated with a $100 increase in price.
After fitting a multiple regression model, an analyst examines a plot of the residuals versus one of the independent variables, $X_1$ . The plot reveals that the points form a distinct curved (U-shaped) pattern, centered horizontally around the residual=0 line. What does this pattern most strongly suggest about the model?
Answer: The assumption of a linear relationship between $X_1$ and the dependent variable is violated. A systematic pattern, such as a curve, in a plot of residuals versus a predictor variable indicates that the model is not capturing the relationship correctly. A U-shaped pattern specifically suggests that the relationship between $X_1$ and the dependent variable is non-linear and that a quadratic term (e.g., $X_1^2$ ) might be missing from the model. This is a violation of the linearity assumption. Heteroscedasticity would show a change in the vertical spread of the points (a cone shape), not a curve.
A sports analyst is modeling the number of wins for a baseball team. The predictors are Runs Scored (RS) and Runs Allowed (RA). Both variables are known to be highly significant predictors. The analyst then adds a third predictor, Run Differential (RD), which is calculated as RD = RS - RA. The new model is Wins ~ RS + RA + RD. Which of the following is the most likely outcome for the coefficients in this new model?
Answer: The standard errors of the coefficients will become extremely large, and the signs of the coefficients may be counter-intuitive. This is a case of perfect multicollinearity because one predictor (RD) is a perfect linear combination of two other predictors (RS and RA). In practice, statistical software will either drop one of the variables or fail to run. Conceptually, if the model were to be estimated, the standard errors of the coefficients for RS, RA, and RD would become infinitely large. This makes the coefficient estimates completely unstable and unreliable. It's common in such situations for coefficients to have illogical signs (e.g., wins decreasing with runs scored) because the model cannot disentangle their individual effects.
A financial analyst develops two regression models to predict a company's stock return, using a sample of 100 observations. * Model A: Uses 3 predictors and has an R-squared of 0.650. * Model B: Uses the same 3 predictors as Model A plus 2 additional predictors that are only weakly correlated with the stock return. If Model B has an R-squared of 0.660, which statement is the most accurate comparison of the two models?
Answer: Model A is likely preferable because its adjusted R-squared is probably higher than Model B's. Adjusted R-squared is used to compare models with different numbers of predictors. It penalizes the model for adding predictors that do not significantly improve the fit. While R-squared will always increase (or stay the same) when a new predictor is added, adjusted R-squared will only increase if the added predictor improves the model more than would be expected by chance. A small increase in R-squared (from 0.650 to 0.660) from adding two predictors is unlikely to overcome the penalty, meaning Model A's adjusted R-squared is likely higher than Model B's, making it the more parsimonious and preferable model.
A marketing analyst runs a multiple regression to predict monthly sales. The model includes several predictors related to advertising spend. The overall model has a high adjusted R-squared of 0.85 and a statistically significant F-test (p < 0.001). However, none of the individual predictor variables have statistically significant t-statistics (all p > 0.10). Which of the following is the most likely explanation for this outcome?
Answer: High multicollinearity exists among the predictor variables. This is a classic symptom of severe multicollinearity. When predictors are highly correlated, the model as a whole can have strong explanatory power (high R-squared, significant F-test), but the standard errors of the individual coefficients become inflated. This makes it difficult to disentangle the unique contribution of each predictor, leading to insignificant t-tests. The model can't tell which of the correlated predictors is 'responsible' for the effect.
A human resources director uses multiple regression to predict employee satisfaction scores (1-100 scale) with the model: Satisfaction = 35 + 8(Training_Hours) + 12(Salary_Level) + 5(Team_Size), where Training_Hours is measured annually, Salary_Level is coded 1-5, and Team_Size is number of team members. What does the coefficient of Team_Size indicate about the relationship?
Answer: A 5-point satisfaction increase per team member, assuming linear relationship and no interaction effects with other variables The coefficient 5 for Team_Size means that, holding other variables constant, each additional team member is associated with a 5-point increase in satisfaction, assuming the relationship is linear and there are no interaction effects. Option A is too absolute by saying 'across all team configurations' without acknowledging assumptions. Option B incorrectly interprets the coefficient magnitude as indicating 'diminishing returns' - the coefficient represents a constant marginal effect, not diminishing returns. Option D confuses the coefficient with explained variance - coefficients don't directly indicate percentage of variance explained.
An operations researcher develops a model: Productivity = α + β₁(Experience) + β₂(Training) + β₃(Equipment_Age) + ε, using data from 60 workers. The F-statistic for overall model significance is 12.8. If the critical F-value at 5% significance level is 2.76, and individual t-tests show that only Experience and Training are significant, what conclusion is most appropriate?
Answer: The model is overall significant and useful for prediction, though Equipment_Age may not contribute meaningfully The F-statistic of 12.8 exceeds the critical value of 2.76, so the overall model is statistically significant, meaning at least one predictor is related to productivity. However, Equipment_Age being individually non-significant suggests it may not add meaningful explanatory power. This situation is common and doesn't contradict overall significance. Option A is too hasty - variable removal should consider theory and adjusted R². Option C is wrong because individual non-significance doesn't contradict overall significance when other variables are significant. Option D is wrong because overall significance doesn't automatically justify retaining non-significant individual variables.
A simple linear regression model is fit to a dataset. Later, a single data point is added. This new point has a value for the independent variable that is far from the mean of the other data points, but it lies very close to the originally fitted regression line. Which of the following is the most likely impact of adding this point and refitting the model?
Answer: The R-squared value will likely increase, and the standard errors of the coefficients will likely decrease. This point is a 'good' leverage point. Because its x-value is extreme, it has high leverage. Because it lies close to the existing regression line, it confirms the trend observed in the rest of the data. Adding such a point will anchor the regression line more firmly, reducing the uncertainty in the slope estimate and thus decreasing the standard errors of the coefficients. By extending the range of X and fitting the pattern well, it will also typically increase the R-squared value.
An economist estimates a multiple regression model: GDP_Growth = β₀ + β₁(Investment_Rate) + β₂(Education_Index) + β₃(Trade_Openness) + ε. The model achieves R² = 0.72 with 45 observations. When a fourth variable (Infrastructure_Quality) is added, R² increases to 0.74. What conclusion about model improvement is most appropriate?
Answer: The improvement should be evaluated using adjusted R² or F-test since R² always increases with additional variables R² always increases (or stays the same) when variables are added, regardless of whether they truly improve the model. A 0.02 increase from 0.72 to 0.74 with one additional variable and 45 observations may not represent genuine improvement. Adjusted R² or an F-test for the added variable would properly account for the loss of degrees of freedom. Option A is wrong because R² increase alone doesn't indicate significance. Option B assumes the increase is meaningful without proper testing. Option D overstates the practical significance of a 0.02 increase.
A retail analyst builds a model to predict monthly store sales using: Sales = β₀ + β₁(Store_Size) + β₂(Foot_Traffic) + β₃(Local_Income) + ε. After estimation, the analyst finds that residual plots show increasing variance as predicted values increase, and the Durbin-Watson statistic is 0.85. Which model assumption violations require attention?
Answer: Both heteroscedasticity and autocorrelation, requiring corrections for variance patterns and temporal dependencies When you encounter regression diagnostics, you need to systematically evaluate multiple assumption violations that can occur simultaneously. This question tests your ability to interpret both residual patterns and diagnostic statistics to identify specific violations. The key evidence points to two distinct problems. First, "increasing variance as predicted values increase" from residual plots is the textbook definition of heteroscedasticity—where error variance isn't constant across all levels of predicted values. Second, the Durbin-Watson statistic of 0.85 signals positive autocorrelation, since values near 0 indicate strong positive correlation in residuals (the benchmark value of 2.0 indicates no autocorrelation). Option D correctly identifies both violations. Heteroscedasticity makes standard errors unreliable, while autocorrelation violates the independence assumption, both requiring specific corrections. Option A misses the autocorrelation problem entirely. While it correctly identifies heteroscedasticity and appropriate remedies (weighted least squares, robust standard errors), ignoring the low Durbin-Watson statistic overlooks a serious violation. Option B makes the opposite error—recognizing autocorrelation but missing heteroscedasticity. The clear pattern of increasing variance in residual plots cannot be ignored. Option C incorrectly interprets the diagnostic evidence. Neither increasing residual variance nor low Durbin-Watson statistics indicate multicollinearity, which would show up in correlation matrices or variance inflation factors, not these particular patterns. Study tip: Always check multiple diagnostics systematically. Regression problems rarely come in isolation—heteroscedasticity and autocorrelation frequently appear together, especially in time-series business data like monthly sales.
An analyst is concerned about multicollinearity in a regression model with four predictors: $X_1, X_2, X_3, X_4$ . The Variance Inflation Factors (VIFs) for the predictors are calculated as follows: VIF( $X_1$ ) = 1.2, VIF( $X_2$ ) = 8.9, VIF( $X_3$ ) = 15.4, and VIF( $X_4$ ) = 1.5. Based on these VIFs, what is the most pressing concern for the analyst?
Answer: The standard error for the coefficient of $X_3$ is significantly inflated, making its individual effect difficult to estimate precisely. The Variance Inflation Factor (VIF) measures how much the variance of an estimated regression coefficient is increased because of multicollinearity. A common rule of thumb is that a VIF greater than 10 (as for $X_3$ ) indicates a problematic level of multicollinearity. The direct consequence is not bias in the coefficient estimate but a large inflation of its standard error, which widens the confidence interval and makes the estimate unreliable and unstable.
A marketing analyst models the natural logarithm of sales (ln(Sales)) as a function of advertising spending (AdSpend, measured in thousands of dollars). The fitted equation is: $\widehat{\ln(\text{Sales})} = 5.2 + 0.04 \times \text{AdSpend}$ How should the coefficient 0.04 be interpreted?
Answer: For each additional $1,000 in advertising spending, sales are expected to increase by approximately 4%. This is a log-level model. The interpretation for a coefficient $\beta_1$ in such a model is that a one-unit change in the independent variable is associated with a $100 \times \beta_1$ % change in the dependent variable. Here, AdSpend is in thousands of dollars, so a one-unit change is an additional $1,000. The coefficient is 0.04. Therefore, an additional $1,000 in advertising spend is associated with an approximate $100 \times 0.04 = 4$ % increase in sales.
A multiple regression model with five predictor variables is fitted to a dataset. The overall F-statistic for the model is 25.6 with a p-value of < 0.001. The t-statistics for the five predictors have p-values of 0.02, 0.04, 0.35, 0.41, and 0.58, respectively. What is the most appropriate conclusion based on this information?
Answer: There is strong evidence that the set of five predictor variables, as a group, is useful for predicting the response. The overall F-test assesses the null hypothesis that all regression coefficients are simultaneously equal to zero ( $H_0: \beta_1 = \beta_2 = \dots = \beta_5 = 0$ ). A small p-value for the F-test (like < 0.001) provides strong evidence to reject this null hypothesis. This means that at least one of the predictor variables is linearly related to the dependent variable. It validates the usefulness of the model as a whole, even if not all individual predictors are significant.
A regression model is used to predict household electricity consumption based on income. After fitting the model, a plot of the residuals versus the predicted values shows a distinct cone shape, with the spread of the residuals increasing as the predicted consumption increases. What is the primary implication of this finding?
Answer: The standard errors of the regression coefficients are unreliable, making hypothesis tests and confidence intervals invalid. A cone-shaped pattern in the residual plot (residuals vs. predicted values) is the classic sign of heteroscedasticity, which means the variance of the errors is not constant. While heteroscedasticity does not cause bias in the coefficient estimates themselves, it violates the assumption of constant variance. This leads to incorrect (typically underestimated) standard errors for the coefficients, which in turn makes t-tests, p-values, and confidence intervals unreliable and invalid.
A researcher models employee salary (Y) using years of experience ( $X_1$ ). The estimated model is $\hat{Y} = b_0 + b_1 X_1$ , and $b_1$ is found to be positive and significant. The researcher realizes that "innate ability" ( $X_2$ ), a relevant predictor of salary, was omitted from the model. It is known that innate ability is positively correlated with salary and also positively correlated with years of experience (more able people may stay in their field longer). What is the likely effect of omitting "innate ability" on the estimated coefficient $b_1$ ?
Answer: The coefficient $b_1$ is likely overestimated. This is a case of omitted variable bias. The bias in $b_1$ is determined by the product of two effects: the effect of the omitted variable (ability) on the dependent variable (salary), and the correlation between the omitted variable (ability) and the included variable (experience). Here, ability has a positive effect on salary, and ability is positively correlated with experience. The product of these two positive effects is positive, meaning the estimated coefficient $b_1$ has a positive bias. Therefore, $b_1$ is likely overestimated.
To analyze the relationship between a company's advertising expenditures and its revenue, an economist fits a log-log regression model using data from 50 different companies: $\widehat{\ln(\text{Revenue})} = 2.5 + 1.1 \times \ln(\text{Advertising})$ How is the coefficient 1.1 correctly interpreted in this context?
Answer: A 1% increase in advertising expenditure is associated with an estimated 1.1% increase in revenue. In a log-log regression model (where both the dependent and independent variables are in natural logarithms), the slope coefficient represents the elasticity. Elasticity is the ratio of the percentage change in the dependent variable for a percentage change in the independent variable. Therefore, a coefficient of 1.1 means that a 1% increase in the independent variable (Advertising) is associated with an estimated 1.1% increase in the dependent variable (Revenue).
A supply chain manager uses multiple regression to predict delivery time (in days) based on distance (X₁, in miles), package weight (X₂, in pounds), and weather severity index (X₃, scale 1-10). The model equation is: Time = 2.1 + 0.005X₁ + 0.02X₂ + 0.8X₃. If the correlation between distance and weather severity is 0.65, what interpretation concern should the manager prioritize?
Answer: Multicollinearity may inflate standard errors and make coefficient estimates unreliable for policy decisions A correlation of 0.65 between predictors indicates moderate to strong correlation, which raises multicollinearity concerns. Multicollinearity inflates standard errors, making coefficient estimates less reliable and hypothesis tests less powerful. Option B is wrong because correlation between predictors doesn't necessarily indicate need for interaction terms. Option C is wrong because 0.65 correlation, while concerning, doesn't automatically mean one variable should be removed. Option D is wrong because correlation between predictors relates to multicollinearity, not heteroscedasticity (which concerns error variance patterns).
An analyst models the daily number of customers at a restaurant. One predictor is "Season," a categorical variable. The model uses "Winter" as the reference category and includes dummy variables for "Spring," "Summer," and "Fall." The fitted model equation is: $\widehat{\text{Customers}} = 180 - 20(\text{Spring}) + 50(\text{Summer}) + 10(\text{Fall}) + \dots$ How should the coefficient for "Summer" (50) be interpreted?
Answer: The model predicts that, holding other factors constant, there are 50 more customers on an average day in Summer compared to Winter. In a multiple regression with dummy variables, the coefficient for a particular category represents the estimated difference in the mean response between that category and the reference (or baseline) category, holding all other variables constant. In this model, "Winter" is the reference category. Therefore, the coefficient of 50 for "Summer" means that the model predicts an average of 50 more customers per day in Summer than in Winter.
An agronomist models crop yield (in bushels per acre) based on the amount of fertilizer used (F, in pounds per acre). To capture a potential non-linear effect, both F and its square, $F^2$ , are included. The fitted model is: $\widehat{\text{Yield}} = 10 + 5.2F - 0.3F^2$ What does this model imply about the relationship between fertilizer and yield?
Answer: The effect of additional fertilizer on yield is initially positive but diminishes and eventually becomes negative as more fertilizer is applied. This is a quadratic model. The positive coefficient on F (5.2) indicates that at low levels of fertilizer, increasing F leads to an increase in yield. The negative coefficient on $F^2$ (-0.3) indicates that the relationship is concave down (an inverted U-shape). This means the positive effect of fertilizer diminishes as more is applied, eventually reaching a peak, after which additional fertilizer will lead to a decrease in yield. This is known as the law of diminishing returns.

Start this quiz