Linear Regression via Least Squares - Linear Algebra

  1. Which point does a least-squares line y=β0+β1xy = \beta_0 + \beta_1x (with intercept) always pass through?
    (xˉ,yˉ)(\bar{x}, \bar{y}), the point of means of the data.
  2. With design matrix XX and observations y\mathbf{y}, what system gives the regression coefficients β\boldsymbol{\beta}?
    XTXβ=XTyX^TX\boldsymbol{\beta} = X^T\mathbf{y}.
  3. What is the sum of the residuals for a least-squares line that includes an intercept?
    00. The residual vector is orthogonal to the column of ones in the design matrix.
  4. Why can y=β0+β1sin⁡xy = \beta_0 + \beta_1\sin x be fit by linear least squares?
    The model is linear in the parameters β0,β1\beta_0, \beta_1; the design matrix rows are [1,sin⁡xi][1, \sin x_i].
  5. In linear regression, what is the residual of data point ii?
    The observed value minus the predicted value: yi−(β0+β1xi)y_i - (\beta_0 + \beta_1x_i).
  6. For the model y=β0+β1x+β2x2y = \beta_0 + \beta_1x + \beta_2x^2, what is row ii of the design matrix?
    [1,xi,xi2][1, x_i, x_i^2].
  7. Fit y=βxy = \beta x (no intercept) to (1,2)(1, 2) and (2,3)(2, 3) by least squares. What is β\beta?
    85\frac{8}{5}. With one column, β=∑xiyi∑xi2=2+61+4\beta = \frac{\sum x_iy_i}{\sum x_i^2} = \frac{2 + 6}{1 + 4}.
  8. Which quantity does the least-squares regression line minimize?
    The sum of the squared residuals, ∑i(yi−β0−β1xi)2\sum_i (y_i - \beta_0 - \beta_1x_i)^2.
  9. For fitting y=β0+β1xy = \beta_0 + \beta_1x to data (xi,yi)(x_i, y_i), what is row ii of the design matrix?
    [1,xi][1, x_i]. The 11 multiplies the intercept β0\beta_0.
  10. What is the fitted intercept when y=β0+β1xy = \beta_0 + \beta_1x is fit to (−1,0)(-1, 0), (0,1)(0, 1), (1,5)(1, 5)?
    β0=2\beta_0 = 2. Since xˉ=0\bar{x} = 0, the line passes through (0,yˉ)(0, \bar{y}), and yˉ=2\bar{y} = 2.
  11. What least-squares line fits (1,2)(1, 2), (2,4)(2, 4), (3,6)(3, 6)?
    y=2xy = 2x. The points lie exactly on this line, so every residual is 00.
  12. What is the design matrix for fitting y=β0+β1xy = \beta_0 + \beta_1x to (0,1)(0, 1), (1,3)(1, 3), (2,4)(2, 4)?
    [[1,0],[1,1],[1,2]][[1, 0], [1, 1], [1, 2]], with observation vector (1,3,4)(1, 3, 4).

Study these flashcards