Gauss-Markov Theorem & BLUE - Statistics Graduate Level

1. Which statement correctly distinguishes the Gauss–Markov conclusion from likelihood-based optimality?
   Answer: OLS remains BLUE, but the theorem does not compare it with the nonlinear least absolute deviations estimator. When you encounter a question comparing the Gauss–Markov theorem to likelihood-based optimality, ask yourself: what does each framework actually guarantee, and within what class of estimators? The Gauss–Markov theorem states that OLS is the Best Linear Unbiased Estimator (BLUE) whenever errors have mean zero, constant variance, and are uncorrelated — no distributional assumption is required. Under the Laplace model, these conditions hold, so OLS remains BLUE. However, "best" here means minimum variance among all linear unbiased estimators. The least absolute deviations (LAD) estimator is nonlinear in the response variable, so it simply falls outside the class Gauss–Markov considers. The theorem is silent about whether LAD could outperform OLS — and under Laplace errors, LAD actually does achieve lower asymptotic variance. Both facts coexist without contradiction, making B correct. A is wrong because it misrepresents the theorem's requirements. Gauss–Markov does not require normality — that's a common misconception. Normality enters only when you want OLS to be the overall MLE or to derive exact finite-sample distributions. C is wrong on two counts: LAD is not BLUE (it's nonlinear), and MLEs do not generally achieve minimum finite-sample variance. The Cramér–Rao bound is an asymptotic result, and MLEs can be biased or inefficient in small samples. D is wrong because sharing the same estimand doesn't constrain covariance matrices to be equal. Different estimators of the same parameter routinely have different variances. Study tip: Always track the class of estimators a theorem applies to. Gauss–Markov's "best" is restricted to linear estimators — any nonlinear competitor lives outside its scope entirely.
2. What does the Gauss–Markov theorem imply about this claim?
   Answer: The claim is impossible because the displayed covariance difference is not positive semidefinite. Whenever you see a question invoking the Gauss–Markov theorem, your central task is verifying whether a proposed estimator is consistent with the theorem's conclusion — namely, that the OLS estimator $\widehat{\beta}$ is BLUE (Best Linear Unbiased Estimator). The operative word is "best": for any other linear unbiased estimator $T$, the matrix $\operatorname{Var}(T) - \operatorname{Var}(\widehat{\beta})$ must be positive semidefinite (PSD). This is the matrix analog of saying $T$ has "at least as much variance" as OLS in every direction. To check PSD, you can examine the leading principal minors. The proposed difference matrix is $D = \sigma^2\begin{pmatrix}1 & 0.8 \\ 0.8 & 0.5\end{pmatrix}.$ The $1\times 1$ minor is $1 > 0$, but the determinant is $\sigma^2(1)(0.5) - \sigma^2(0.8)^2 = \sigma^2(0.5 - 0.64) = -0.14\sigma^2 < 0$. A negative determinant means $D$ is indefinite, not PSD. This makes the claim impossible — no valid linear unbiased estimator can produce this covariance difference. So C is correct. A is wrong because having positive diagonal entries is necessary but not sufficient for PSD. Off-diagonal entries matter, and here the $(1,2)$ entry is large enough to destroy positive semidefiniteness. B falls into the same trap: a positive trace only tells you the sum of eigenvalues is positive, not that all eigenvalues are non-negative. D introduces an irrelevant condition — Gauss–Markov applies regardless of whether marginal variances are equal or unequal. Study tip: On matrix-based exam questions, never judge definiteness by diagonal entries or trace alone — always check the determinant (or all eigenvalues). This is a classic trap.
3. Why does this covariance improvement not contradict the Gauss–Markov theorem?
   Answer: The restricted estimator is unbiased only on the restricted parameter space, whereas the usual theorem compares estimators unbiased on the full space. Whenever you see a question about restricted versus unrestricted estimators, anchor yourself to the exact conditions under which Gauss–Markov applies: it guarantees that OLS is the Best Linear Unbiased Estimator (BLUE) among all estimators that are unbiased across the entire parameter space $\mathbb{R}^k$. The restricted least-squares estimator imposes $R\beta = r$, which means it is designed to be unbiased only when that constraint is actually true — i.e., on the restricted subspace. If you moved to a world where $R\beta \neq r$, the restricted estimator would be biased. Gauss–Markov never claimed to crown OLS over estimators that are only conditionally unbiased. The two estimators are simply competing on different turf: OLS wins the BLUE title over the full unbiasedness class, while the restricted estimator wins efficiency within the narrower class of estimators unbiased on the restricted space. That's why A is correct — the theorem's comparison class and the restricted estimator's unbiasedness class don't overlap in the right way to create a contradiction. B is wrong because the restricted estimator is linear in $y$ — it equals $\hat{\beta}_{OLS}$ minus a linear correction term, so linearity is not the issue. C is wrong because both estimators target the same $\beta$; they don't concern "unrelated parameters." D is wrong and inverts the logic — the theorem doesn't exclude such estimators by fiat; rather, those estimators simply fall outside the unbiasedness condition the theorem requires. As a study tip, always parse Gauss–Markov carefully: "best" means minimum variance, "linear" means linear in $y$, and "unbiased" means globally unbiased. Restrictions on any one of these words can dissolve apparent contradictions on exam questions.
4. Which assessment of the reported analysis is most accurate?
   Answer: OLS remains unbiased but is not generally BLUE; robust standard errors do not restore coefficient efficiency. When a question involves heteroskedasticity and OLS, you should immediately organize your thinking around three separate properties: unbiasedness, efficiency (BLUE), and inference validity. These are independent, and confusing them is exactly what the distractors exploit. Under the Gauss-Markov theorem, OLS is BLUE only when errors are homoskedastic. The "B" in BLUE — Best — means minimum variance among linear unbiased estimators. Unbiasedness, however, requires only that $E(\varepsilon \mid X) = 0$, which the passage explicitly grants. So OLS remains unbiased even with heteroskedasticity; it simply loses its efficiency claim because a feasible GLS estimator would have lower variance. Reporting heteroskedasticity-consistent (robust) standard errors corrects the inference — it gives you valid hypothesis tests and confidence intervals — but it does nothing to change the coefficient estimates themselves or their sampling variance. The OLS estimator is still inefficient. Answer D captures all of this precisely: unbiased, not BLUE, and robust SEs don't fix efficiency. A is wrong because robust standard errors correct the estimated covariance matrix used for inference; they do not alter the OLS estimator's actual sampling distribution or restore the BLUE property. Calling OLS "BLUE" post-correction is a category error. B is wrong because heteroskedasticity alone — without a violated mean assumption — does not cause bias. This is a very common misconception worth memorizing explicitly. C is wrong in claiming OLS remains BLUE. Efficiency is lost under heteroskedasticity; robust SEs just prevent you from drawing wrong conclusions from that inefficient estimator. Study tip: On efficiency questions, always ask yourself: which assumption does Gauss-Markov require for "Best"? Homoskedasticity. Violate it, and OLS loses BLUE status — full stop.
5. What is $\operatorname{Var}(T),$ and what feature of the Gauss–Markov argument explains the result?
   Answer: It is $5\sigma^2/6,$ because the sample mean and the residual contrast are uncorrelated. When you encounter a question about linear estimators and variance, your first instinct should be to compute directly using independence, then connect the result to the Gauss–Markov theorem's logic about efficient estimation. Start by writing $T = \bar{Y} + \frac{Y_1 - Y_2}{2}$, where $\bar{Y} = \frac{Y_1 + Y_2 + Y_3}{3}$. Expanding the coefficients on each $Y_i$: $T = \left(\frac{1}{3} + \frac{1}{2}\right)Y_1 + \left(\frac{1}{3} - \frac{1}{2}\right)Y_2 + \frac{1}{3}Y_3 = \frac{5}{6}Y_1 - \frac{1}{6}Y_2 + \frac{1}{3}Y_3.$ Since the $Y_i$ are independent, $\operatorname{Var}(T) = \sigma^2\left[\left(\frac{5}{6}\right)^2 + \left(\frac{1}{6}\right)^2 + \left(\frac{1}{3}\right)^2\right] = \sigma^2\left[\frac{25}{36} + \frac{1}{36} + \frac{4}{36}\right] = \frac{30}{36}\sigma^2 = \frac{5\sigma^2}{6}.$ This confirms D. The Gauss–Markov insight is that the contrast $\frac{Y_1 - Y_2}{2}$ has mean zero (preserving unbiasedness) and is uncorrelated with $\bar{Y}$, yet it inflates variance beyond $\sigma^2/3$, illustrating precisely why OLS (the sample mean here) is BLUE — any perturbation by such a contrast strictly increases variance. Choice A is wrong because adding a zero-mean contrast does inflate the variance; Gauss–Markov says variance increases, not stays constant. Choice B incorrectly claims positive covariance between $\bar{Y}$ and the contrast — you can verify the covariance is actually zero, not positive. Choice C invents a replacement mechanism that has no basis in the actual coefficient arithmetic. Your study tip: always expand a linear estimator into explicit coefficients on each $Y_i$ before computing variance. It bypasses covariance questions entirely and reveals the true structure.
6. What are the value and variance of the BLUE of $\mu$?
   Answer: The value is $20/7,$ and its variance is $36/49.$ When you see a model with unequal error variances, your instinct should jump to weighted least squares — specifically, the Gauss-Markov theorem tells you the Best Linear Unbiased Estimator (BLUE) assigns weights inversely proportional to each observation's variance. Here, $\text{Var}(\varepsilon_i) = \sigma_i^2 = 1, 4, 9,$ so the weights are $w_i = 1/\sigma_i^2 = 1, 1/4, 1/9.$ The BLUE of $\mu$ is the weighted average: $\hat{\mu} = \frac{\sum w_i Y_i}{\sum w_i} = \frac{(1)(2) + (1/4)(4) + (1/9)(8)}{1 + 1/4 + 1/9}.$ Computing the numerator: $2 + 1 + 8/9 = 3 + 8/9 = 35/9.$ The denominator: $1 + 1/4 + 1/9 = 36/36 + 9/36 + 4/36 = 49/36.$ So $\hat{\mu} = \frac{35/9}{49/36} = \frac{35}{9} \cdot \frac{36}{49} = \frac{35 \cdot 4}{49} = \frac{140}{49} = \frac{20}{7}.$ Its variance is $\frac{1}{\sum w_i} = \frac{1}{49/36} = \frac{36}{49}.$ This confirms B. Choice A likely comes from using a simple unweighted average $(2+4+8)/3 = 14/3$ — ignoring the heteroscedasticity entirely. Choice C results from incorrectly weighting by $\sigma_i$ rather than $1/\sigma_i^2$. Choice D resembles a harmonic-mean-style calculation error in combining the weights. The key study tip: for any BLUE problem with known unequal variances, always write down $w_i = 1/\sigma_i^2$ first, then the variance of the estimator is simply $1/\sum w_i$ — this single formula generates both the estimate and its variance cleanly.
7. Which conclusion most accurately applies the Gauss–Markov theorem in this setting?
   Answer: Conditional on $X,$ OLS is BLUE among linear conditionally unbiased estimators whose coefficients may depend on $X.$ When you see a question about the Gauss–Markov theorem with random regressors, the key is recognizing that the theorem's power comes from conditioning on $X$, not from treating $X$ as fixed by assumption. Given $E(\varepsilon \mid X) = 0$ and $\operatorname{Var}(\varepsilon \mid X) = \sigma^2 I$, the classical Gauss–Markov conditions hold conditional on the realized $X$. Since the realized design matrix has full column rank, the OLS estimator $\hat{\beta} = (X^\top X)^{-1}X^\top Y$ is well-defined. Conditionally on $X$, OLS is unbiased and achieves the minimum variance among all linear estimators whose coefficient matrices can depend on $X$ — that is, estimators of the form $CY$ where $C$ may be a function of $X$. This makes B the correct and precise statement. A is wrong because random regressors do not block conditional analysis. The whole point of conditioning on $X$ is to treat it as fixed temporarily, which is entirely valid under the law of iterated expectations. C overstates the result. Gauss–Markov guarantees BLUE only within the class of linear estimators. It says nothing about nonlinear estimators, which can sometimes achieve lower variance (e.g., under non-Gaussian errors, MLE may outperform OLS). D is wrong for the same reason as A — conditioning on $X$ is exactly what enables the efficiency comparison. The randomness of $X$ is handled by the conditioning argument, not avoided. Your study tip: always track the class of estimators over which "best" is claimed — linear vs. all unbiased, and conditional vs. unconditional. Gauss–Markov is specifically about linear, conditionally unbiased estimators.
8. Which condition is sufficient for ordinary least squares to remain BLUE even though $\Sigma$ need not be proportional to the identity matrix?
   Answer: There exists a matrix $C$ such that $\Sigma X=XC.$ Whenever you encounter a question about OLS efficiency under non-spherical errors, your anchor should be the Gauss-Markov theorem and its generalizations. The theorem guarantees OLS is BLUE when $\Sigma = \sigma^2 I$, but a weaker condition can preserve this property. The key insight is that OLS and GLS produce identical estimates whenever $X\beta$ lies in the column space that is "unchanged" by $\Sigma$. Formally, if there exists a matrix $C$ such that $\Sigma X = XC$, then $\Sigma$ maps the column space of $X$ back into itself. Under this condition, the GLS estimator $(X^\top \Sigma^{-1} X)^{-1} X^\top \Sigma^{-1} Y$ simplifies algebraically to the OLS estimator $(X^\top X)^{-1} X^\top Y$, making OLS BLUE. This is precisely condition A, and it's the classical result due to Kruskal (1968). Choice B is wrong because $X^\top \Sigma X = X^\top X$ has no direct algebraic justification for collapsing GLS to OLS — it conflates the roles of $\Sigma$ and $\Sigma^{-1}$ in the GLS formula and is not a recognized sufficient condition. Choice C is a common misconception: having an intercept handles certain bias problems but does nothing to restore efficiency when heteroskedasticity is present. Unequal diagonal entries of $\Sigma$ mean OLS is generally inefficient regardless of an intercept. Choice D confuses nonsingularity (which is already assumed via positive-definiteness) with a condition on efficiency. Equal diagonal entries of $X^\top \Sigma^{-1} X$ carry no special implication for OLS being BLUE. Your study tip: memorize the condition $\Sigma X = XC$ as the elegant "invariant subspace" criterion — it's the precise algebraic test for when you can skip GLS entirely.
9. Which coefficient vector produces the BLUE of $\beta$?
   Answer: $(a_1,a_2,a_3)=(1/14,1/7,3/14)$ Whenever you see a question about the Best Linear Unbiased Estimator (BLUE), invoke the Gauss-Markov theorem: among all linear unbiased estimators, the OLS estimator has minimum variance. For $T = \sum a_i Y_i$ to be unbiased for $\beta$, we need $E[T] = \beta \sum a_i x_i = \beta$, so the unbiasedness constraint is $\sum a_i x_i = a_1(1) + a_2(2) + a_3(3) = 1$. Among all vectors satisfying this, OLS picks the one minimizing variance $\text{Var}(T) = \sigma^2 \sum a_i^2$. The OLS estimator here is $\hat{\beta} = \frac{\sum x_i Y_i}{\sum x_i^2} = \frac{1\cdot Y_1 + 2\cdot Y_2 + 3\cdot Y_3}{1^2+2^2+3^2} = \frac{Y_1 + 2Y_2 + 3Y_3}{14}$, giving coefficients $(1/14, 2/14, 3/14) = (1/14, 1/7, 3/14)$. This is answer D, the BLUE by Gauss-Markov. Check unbiasedness: $\frac{1}{14}(1) + \frac{1}{7}(2) + \frac{3}{14}(3) = \frac{1+4+9}{14} = 1$ ✓ Now the distractors: A gives equal weights $(1/6, 1/6, 1/6)$, which fails unbiasedness — $\frac{1}{6}(1+2+3) = 1$ actually passes, but its variance $\sigma^2/6$ is larger than OLS variance $\sigma^2/14$. B fails the unbiasedness check: $\frac{1}{3}(1)+0(2)+\frac{2}{9}(3) = \frac{1}{3}+\frac{2}{3}=1$, but $\sum a_i^2 > 1/14$, so it's unbiased but not minimum variance. C fails unbiasedness entirely: $\frac{1}{3}(1)+\frac{1}{3}(2)+0(3) = 1 \neq 1$... actually equals $1$, but again its variance exceeds that of D. Your key takeaway: always derive BLUE via the OLS formula $\hat{\beta} = \frac{\sum x_i Y_i}{\sum x_i^2}$ for simple no-intercept models, and verify candidates by checking both the unbiasedness constraint and comparing $\sum a_i^2$.

Start this quiz