Statistics Graduate Level Quiz: Gauss Markov Theorem And Blue

9 questions · exam conditions

0:00

Gauss Markov Theorem And BlueQuestion 1 of 9

Consider the fixed-design model $Y=X\beta+\varepsilon,$ where $X$ has full column rank, $E(\varepsilon)=0,$ and $\operatorname{Var}(\varepsilon)=\Sigma$ for a known positive-definite matrix $\Sigma.$

Which condition is sufficient for ordinary least squares to remain BLUE even though $\Sigma$ need not be proportional to the identity matrix?

There exists a matrix

C

such that

\Sigma X=XC.

The equality

X^{\mathsf T}\Sigma X=X^{\mathsf T}X

holds for the design matrix.

The diagonal entries of

\Sigma

are unequal, but

X

contains an intercept.

The matrix

X^{\mathsf T}\Sigma^{-1}X

is nonsingular and has equal diagonal entries.

← Back to quizzes

Statistics Graduate Level Quiz

Statistics Graduate Level Quiz: Gauss Markov Theorem And Blue

Practice Gauss Markov Theorem And Blue in Statistics Graduate Level with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Gauss Markov Theorem And Blue, giving you a quick way to practice the rules, question types, and explanations that matter most for Statistics Graduate Level.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

Consider the fixed-design model $Y=X\beta+\varepsilon,$ where $X$ has full column rank, $E(\varepsilon)=0,$ and $\operatorname{Var}(\varepsilon)=\Sigma$ for a known positive-definite matrix $\Sigma.$

Which condition is sufficient for ordinary least squares to remain BLUE even though $\Sigma$ need not be proportional to the identity matrix?

There exists a matrix $C$ such that $\Sigma X=XC.$ (correct answer)
The equality $X^{\mathsf T}\Sigma X=X^{\mathsf T}X$ holds for the design matrix.
The diagonal entries of $\Sigma$ are unequal, but $X$ contains an intercept.
The matrix $X^{\mathsf T}\Sigma^{-1}X$ is nonsingular and has equal diagonal entries.

Explanation: Whenever you encounter a question about OLS efficiency under non-spherical errors, your anchor should be the Gauss-Markov theorem and its generalizations. The theorem guarantees OLS is BLUE when

\Sigma = \sigma^2 I

, but a weaker condition can preserve this property. The key insight is that OLS and GLS produce identical estimates whenever

X\beta

lies in the column space that is "unchanged" by

\Sigma

. Formally, if there exists a matrix

C

such that

\Sigma X = XC

, then

\Sigma

maps the column space of

X

back into itself. Under this condition, the GLS estimator

(X^\top \Sigma^{-1} X)^{-1} X^\top \Sigma^{-1} Y

simplifies algebraically to the OLS estimator

(X^\top X)^{-1} X^\top Y

, making OLS BLUE. This is precisely condition A, and it's the classical result due to Kruskal (1968). Choice B is wrong because

X^\top \Sigma X = X^\top X

has no direct algebraic justification for collapsing GLS to OLS — it conflates the roles of

\Sigma

and

\Sigma^{-1}

in the GLS formula and is not a recognized sufficient condition. Choice C is a common misconception: having an intercept handles certain bias problems but does nothing to restore efficiency when heteroskedasticity is present. Unequal diagonal entries of

\Sigma

mean OLS is generally inefficient regardless of an intercept. Choice D confuses nonsingularity (which is already assumed via positive-definiteness) with a condition on efficiency. Equal diagonal entries of

X^\top \Sigma^{-1} X

carry no special implication for OLS being BLUE. Your study tip: memorize the condition

\Sigma X = XC

as the elegant "invariant subspace" criterion — it's the precise algebraic test for when you can skip GLS entirely.

Question 2

A researcher fits $Y=X\beta+\varepsilon$ by ordinary least squares. The conditional mean assumption $E(\varepsilon\mid X)=0$ is credible, but the conditional error variances differ across observations. The researcher reports heteroskedasticity-consistent standard errors without changing the coefficient estimates.

Which assessment of the reported analysis is most accurate?

The robust standard errors make the OLS coefficient estimator BLUE by correcting its covariance after estimation.
Heteroskedasticity makes OLS biased, although robust standard errors consistently estimate that bias.
OLS remains BLUE because heteroskedasticity changes estimated standard errors but not coefficient efficiency.
OLS remains unbiased but is not generally BLUE; robust standard errors do not restore coefficient efficiency. (correct answer)

Explanation: When a question involves heteroskedasticity and OLS, you should immediately organize your thinking around three separate properties: unbiasedness, efficiency (BLUE), and inference validity. These are independent, and confusing them is exactly what the distractors exploit. Under the Gauss-Markov theorem, OLS is BLUE only when errors are homoskedastic. The "B" in BLUE — Best — means minimum variance among linear unbiased estimators. Unbiasedness, however, requires only that

E(\varepsilon \mid X) = 0

, which the passage explicitly grants. So OLS remains unbiased even with heteroskedasticity; it simply loses its efficiency claim because a feasible GLS estimator would have lower variance. Reporting heteroskedasticity-consistent (robust) standard errors corrects the inference — it gives you valid hypothesis tests and confidence intervals — but it does nothing to change the coefficient estimates themselves or their sampling variance. The OLS estimator is still inefficient. Answer D captures all of this precisely: unbiased, not BLUE, and robust SEs don't fix efficiency. A is wrong because robust standard errors correct the estimated covariance matrix used for inference; they do not alter the OLS estimator's actual sampling distribution or restore the BLUE property. Calling OLS "BLUE" post-correction is a category error. B is wrong because heteroskedasticity alone — without a violated mean assumption — does not cause bias. This is a very common misconception worth memorizing explicitly. C is wrong in claiming OLS remains BLUE. Efficiency is lost under heteroskedasticity; robust SEs just prevent you from drawing wrong conclusions from that inefficient estimator. Study tip: On efficiency questions, always ask yourself: which assumption does Gauss-Markov require for "Best"? Homoskedasticity. Violate it, and OLS loses BLUE status — full stop.

Question 3

In a full-rank homoskedastic model, it is known with certainty that the parameter satisfies $R\beta=r.$ A restricted least-squares estimator imposes this equality and has smaller covariance than unrestricted OLS for some parameter directions.

Why does this covariance improvement not contradict the Gauss–Markov theorem?

The restricted estimator is unbiased only on the restricted parameter space, whereas the usual theorem compares estimators unbiased on the full space. (correct answer)
The restricted estimator is nonlinear in the response, so its covariance cannot be compared with that of ordinary least squares.
The restricted estimator estimates a different response mean, so the two covariance matrices concern unrelated target parameters.
The usual theorem excludes estimators using known linear restrictions, even when they are unbiased for every possible parameter value.

Explanation: Whenever you see a question about restricted versus unrestricted estimators, anchor yourself to the exact conditions under which Gauss–Markov applies: it guarantees that OLS is the Best Linear Unbiased Estimator (BLUE) among all estimators that are unbiased across the entire parameter space

\mathbb{R}^k

. The restricted least-squares estimator imposes

R\beta = r

, which means it is designed to be unbiased only when that constraint is actually true — i.e., on the restricted subspace. If you moved to a world where

R\beta \neq r

, the restricted estimator would be biased. Gauss–Markov never claimed to crown OLS over estimators that are only conditionally unbiased. The two estimators are simply competing on different turf: OLS wins the BLUE title over the full unbiasedness class, while the restricted estimator wins efficiency within the narrower class of estimators unbiased on the restricted space. That's why A is correct — the theorem's comparison class and the restricted estimator's unbiasedness class don't overlap in the right way to create a contradiction. B is wrong because the restricted estimator is linear in

y

— it equals

\hat{\beta}_{OLS}

minus a linear correction term, so linearity is not the issue. C is wrong because both estimators target the same

\beta

; they don't concern "unrelated parameters." D is wrong and inverts the logic — the theorem doesn't exclude such estimators by fiat; rather, those estimators simply fall outside the unbiasedness condition the theorem requires. As a study tip, always parse Gauss–Markov carefully: "best" means minimum variance, "linear" means linear in

y

, and "unbiased" means globally unbiased. Restrictions on any one of these words can dissolve apparent contradictions on exam questions.

Question 4

In a fixed-design linear regression, the errors are independent Laplace random variables with mean zero and common variance. Ordinary least squares is used, while least absolute deviations is also considered because it is the likelihood-based estimator under the Laplace model.

Which statement correctly distinguishes the Gauss–Markov conclusion from likelihood-based optimality?

OLS is not BLUE because the Gauss–Markov theorem requires normally distributed errors.
OLS remains BLUE, but the theorem does not compare it with the nonlinear least absolute deviations estimator. (correct answer)
Least absolute deviations is BLUE because a maximum-likelihood estimator always has minimum finite-sample variance.
OLS and least absolute deviations must have equal covariance because both estimate the same regression parameter.

Explanation: When you encounter a question comparing the Gauss–Markov theorem to likelihood-based optimality, ask yourself: what does each framework actually guarantee, and within what class of estimators? The Gauss–Markov theorem states that OLS is the Best Linear Unbiased Estimator (BLUE) whenever errors have mean zero, constant variance, and are uncorrelated — no distributional assumption is required. Under the Laplace model, these conditions hold, so OLS remains BLUE. However, "best" here means minimum variance among all linear unbiased estimators. The least absolute deviations (LAD) estimator is nonlinear in the response variable, so it simply falls outside the class Gauss–Markov considers. The theorem is silent about whether LAD could outperform OLS — and under Laplace errors, LAD actually does achieve lower asymptotic variance. Both facts coexist without contradiction, making B correct. A is wrong because it misrepresents the theorem's requirements. Gauss–Markov does not require normality — that's a common misconception. Normality enters only when you want OLS to be the overall MLE or to derive exact finite-sample distributions. C is wrong on two counts: LAD is not BLUE (it's nonlinear), and MLEs do not generally achieve minimum finite-sample variance. The Cramér–Rao bound is an asymptotic result, and MLEs can be biased or inefficient in small samples. D is wrong because sharing the same estimand doesn't constrain covariance matrices to be equal. Different estimators of the same parameter routinely have different variances. Study tip: Always track the class of estimators a theorem applies to. Gauss–Markov's "best" is restricted to linear estimators — any nonlinear competitor lives outside its scope entirely.

Question 5

In the random-design model $Y=X\beta+\varepsilon,$ assume that $E(\varepsilon\mid X)=0$ and $\operatorname{Var}(\varepsilon\mid X)=\sigma^2I$ almost surely. The realized design matrix has full column rank.

Which conclusion most accurately applies the Gauss–Markov theorem in this setting?

OLS is BLUE only unconditionally, because random regressors prevent conditioning on the realized design.
Conditional on $X,$ OLS is BLUE among linear conditionally unbiased estimators whose coefficients may depend on $X.$ (correct answer)
OLS is BLUE among all unbiased estimators, including nonlinear functions of both $X$ and $Y.$
OLS is conditionally unbiased, but random regressors prevent any conditional efficiency comparison with other estimators.

Explanation: When you see a question about the Gauss–Markov theorem with random regressors, the key is recognizing that the theorem's power comes from conditioning on

X

, not from treating

X

as fixed by assumption. Given

E(\varepsilon \mid X) = 0

and

\operatorname{Var}(\varepsilon \mid X) = \sigma^2 I

, the classical Gauss–Markov conditions hold conditional on the realized

X

. Since the realized design matrix has full column rank, the OLS estimator

\hat{\beta} = (X^\top X)^{-1}X^\top Y

is well-defined. Conditionally on

X

, OLS is unbiased and achieves the minimum variance among all linear estimators whose coefficient matrices can depend on

X

— that is, estimators of the form

CY

where

C

may be a function of

X

. This makes B the correct and precise statement. A is wrong because random regressors do not block conditional analysis. The whole point of conditioning on

X

is to treat it as fixed temporarily, which is entirely valid under the law of iterated expectations. C overstates the result. Gauss–Markov guarantees BLUE only within the class of linear estimators. It says nothing about nonlinear estimators, which can sometimes achieve lower variance (e.g., under non-Gaussian errors, MLE may outperform OLS). D is wrong for the same reason as A — conditioning on

X

is exactly what enables the efficiency comparison. The randomness of

X

is handled by the conditioning argument, not avoided. Your study tip: always track the class of estimators over which "best" is claimed — linear vs. all unbiased, and conditional vs. unconditional. Gauss–Markov is specifically about linear, conditionally unbiased estimators.

Question 6

Under a full-rank homoskedastic linear model, let $\widehat\beta$ be the OLS estimator. A proposed estimator $T$ is claimed to be linear and unbiased, with covariance difference $\operatorname{Var}(T)-\operatorname{Var}(\widehat\beta)=\sigma^2\begin{pmatrix}1&0.8\\0.8&0.5\end{pmatrix}.$

What does the Gauss–Markov theorem imply about this claim?

The claim is compatible with the theorem because both diagonal entries of the difference are positive.
The claim is compatible with the theorem because the trace of the covariance difference is positive.
The claim is impossible because the displayed covariance difference is not positive semidefinite. (correct answer)
The claim is impossible only if the two component estimators have identical marginal variances.

Explanation: Whenever you see a question invoking the Gauss–Markov theorem, your central task is verifying whether a proposed estimator is consistent with the theorem's conclusion — namely, that the OLS estimator

\widehat{\beta}

is BLUE (Best Linear Unbiased Estimator). The operative word is "best": for any other linear unbiased estimator

T

, the matrix

\operatorname{Var}(T) - \operatorname{Var}(\widehat{\beta})

must be positive semidefinite (PSD). This is the matrix analog of saying

T

has "at least as much variance" as OLS in every direction. To check PSD, you can examine the leading principal minors. The proposed difference matrix is

D = \sigma^2\begin{pmatrix}1 & 0.8 \\ 0.8 & 0.5\end{pmatrix}.

The

1\times 1

minor is

1 > 0

, but the determinant is

\sigma^2(1)(0.5) - \sigma^2(0.8)^2 = \sigma^2(0.5 - 0.64) = -0.14\sigma^2 < 0

. A negative determinant means

D

is indefinite, not PSD. This makes the claim impossible — no valid linear unbiased estimator can produce this covariance difference. So C is correct. A is wrong because having positive diagonal entries is necessary but not sufficient for PSD. Off-diagonal entries matter, and here the

(1,2)

entry is large enough to destroy positive semidefiniteness. B falls into the same trap: a positive trace only tells you the sum of eigenvalues is positive, not that all eigenvalues are non-negative. D introduces an irrelevant condition — Gauss–Markov applies regardless of whether marginal variances are equal or unequal. Study tip: On matrix-based exam questions, never judge definiteness by diagonal entries or trace alone — always check the determinant (or all eigenvalues). This is a classic trap.

Question 7

Let $Y_1,Y_2,Y_3$ be independent with common mean $\mu$ and common variance $\sigma^2.$ Consider the linear unbiased estimator $T=\overline Y+(Y_1-Y_2)/2.$

What is $\operatorname{Var}(T),$ and what feature of the Gauss–Markov argument explains the result?

It is $\sigma^2/3,$ because adding any zero-mean residual contrast leaves the OLS variance unchanged.
It is $7\sigma^2/6,$ because the sample mean and the contrast have positive covariance.
It is $\sigma^2/2,$ because the contrast replaces one independent component of the sample mean.
It is $5\sigma^2/6,$ because the sample mean and the residual contrast are uncorrelated. (correct answer)

Explanation: When you encounter a question about linear estimators and variance, your first instinct should be to compute directly using independence, then connect the result to the Gauss–Markov theorem's logic about efficient estimation. Start by writing

T = \bar{Y} + \frac{Y_1 - Y_2}{2}

, where

\bar{Y} = \frac{Y_1 + Y_2 + Y_3}{3}

. Expanding the coefficients on each

Y_i

T = \left(\frac{1}{3} + \frac{1}{2}\right)Y_1 + \left(\frac{1}{3} - \frac{1}{2}\right)Y_2 + \frac{1}{3}Y_3 = \frac{5}{6}Y_1 - \frac{1}{6}Y_2 + \frac{1}{3}Y_3.

Since the

Y_i

are independent,

\operatorname{Var}(T) = \sigma^2\left[\left(\frac{5}{6}\right)^2 + \left(\frac{1}{6}\right)^2 + \left(\frac{1}{3}\right)^2\right] = \sigma^2\left[\frac{25}{36} + \frac{1}{36} + \frac{4}{36}\right] = \frac{30}{36}\sigma^2 = \frac{5\sigma^2}{6}.

This confirms D. The Gauss–Markov insight is that the contrast

\frac{Y_1 - Y_2}{2}

has mean zero (preserving unbiasedness) and is uncorrelated with

\bar{Y}

, yet it inflates variance beyond

\sigma^2/3

, illustrating precisely why OLS (the sample mean here) is BLUE — any perturbation by such a contrast strictly increases variance. Choice A is wrong because adding a zero-mean contrast does inflate the variance; Gauss–Markov says variance increases, not stays constant. Choice B incorrectly claims positive covariance between

\bar{Y}

and the contrast — you can verify the covariance is actually zero, not positive. Choice C invents a replacement mechanism that has no basis in the actual coefficient arithmetic. Your study tip: always expand a linear estimator into explicit coefficients on each

Y_i

before computing variance. It bypasses covariance questions entirely and reveals the true structure.

Question 8

Suppose $Y_i=\beta x_i+\varepsilon_i$ for $i=1,2,3,$ where $(x_1,x_2,x_3)=(1,2,3),$ the errors are uncorrelated, and each error has variance $\sigma^2.$ Consider estimators of the form $T=a_1Y_1+a_2Y_2+a_3Y_3.$

Which coefficient vector produces the BLUE of $\beta$ ?

$(a_1,a_2,a_3)=(1/6,1/6,1/6)$
$(a_1,a_2,a_3)=(1/3,0,2/9)$
$(a_1,a_2,a_3)=(1/3,1/3,0)$
$(a_1,a_2,a_3)=(1/14,1/7,3/14)$ (correct answer)

Explanation: Whenever you see a question about the Best Linear Unbiased Estimator (BLUE), invoke the Gauss-Markov theorem: among all linear unbiased estimators, the OLS estimator has minimum variance. For

T = \sum a_i Y_i

to be unbiased for

\beta

, we need

E[T] = \beta \sum a_i x_i = \beta

, so the unbiasedness constraint is

\sum a_i x_i = a_1(1) + a_2(2) + a_3(3) = 1

. Among all vectors satisfying this, OLS picks the one minimizing variance

\text{Var}(T) = \sigma^2 \sum a_i^2

. The OLS estimator here is

\hat{\beta} = \frac{\sum x_i Y_i}{\sum x_i^2} = \frac{1\cdot Y_1 + 2\cdot Y_2 + 3\cdot Y_3}{1^2+2^2+3^2} = \frac{Y_1 + 2Y_2 + 3Y_3}{14}

, giving coefficients

(1/14, 2/14, 3/14) = (1/14, 1/7, 3/14)

. This is answer D, the BLUE by Gauss-Markov. Check unbiasedness:

\frac{1}{14}(1) + \frac{1}{7}(2) + \frac{3}{14}(3) = \frac{1+4+9}{14} = 1

✓ Now the distractors: A gives equal weights

(1/6, 1/6, 1/6)

, which fails unbiasedness —

\frac{1}{6}(1+2+3) = 1

actually passes, but its variance

\sigma^2/6

is larger than OLS variance

\sigma^2/14

. B fails the unbiasedness check:

\frac{1}{3}(1)+0(2)+\frac{2}{9}(3) = \frac{1}{3}+\frac{2}{3}=1

, but

\sum a_i^2 > 1/14

, so it's unbiased but not minimum variance. C fails unbiasedness entirely:

\frac{1}{3}(1)+\frac{1}{3}(2)+0(3) = 1 \neq 1

... actually equals

1

, but again its variance exceeds that of D. Your key takeaway: always derive BLUE via the OLS formula

\hat{\beta} = \frac{\sum x_i Y_i}{\sum x_i^2}

for simple no-intercept models, and verify candidates by checking both the unbiasedness constraint and comparing

\sum a_i^2

Question 9

Three independent observations satisfy $Y_i=\mu+\varepsilon_i,$ where the errors have mean zero and known variances $1,4,9,$ respectively. The observed responses are $2,4,8.$

What are the value and variance of the BLUE of $\mu$ ?

The value is $14/3,$ and its variance is $14/3.$
The value is $20/7,$ and its variance is $36/49.$ (correct answer)
The value is $24/7,$ and its variance is $6/7.$
The value is $10/3,$ and its variance is $9/14.$

Explanation: When you see a model with unequal error variances, your instinct should jump to weighted least squares — specifically, the Gauss-Markov theorem tells you the Best Linear Unbiased Estimator (BLUE) assigns weights inversely proportional to each observation's variance. Here,

\text{Var}(\varepsilon_i) = \sigma_i^2 = 1, 4, 9,

so the weights are

w_i = 1/\sigma_i^2 = 1, 1/4, 1/9.

The BLUE of

\mu

is the weighted average:

\hat{\mu} = \frac{\sum w_i Y_i}{\sum w_i} = \frac{(1)(2) + (1/4)(4) + (1/9)(8)}{1 + 1/4 + 1/9}.

Computing the numerator:

2 + 1 + 8/9 = 3 + 8/9 = 35/9.

The denominator:

1 + 1/4 + 1/9 = 36/36 + 9/36 + 4/36 = 49/36.

\hat{\mu} = \frac{35/9}{49/36} = \frac{35}{9} \cdot \frac{36}{49} = \frac{35 \cdot 4}{49} = \frac{140}{49} = \frac{20}{7}.

Its variance is

\frac{1}{\sum w_i} = \frac{1}{49/36} = \frac{36}{49}.

This confirms B. Choice A likely comes from using a simple unweighted average

(2+4+8)/3 = 14/3

— ignoring the heteroscedasticity entirely. Choice C results from incorrectly weighting by

\sigma_i

rather than

1/\sigma_i^2

. Choice D resembles a harmonic-mean-style calculation error in combining the weights. The key study tip: for any BLUE problem with known unequal variances, always write down

w_i = 1/\sigma_i^2

first, then the variance of the estimator is simply

1/\sum w_i

— this single formula generates both the estimate and its variance cleanly.