Statistics Graduate Level Quiz: Bias Variance And Mse

10 questions · exam conditions

0:00

Bias Variance And MseQuestion 1 of 10

An estimator $T$ of $g(\theta)$ has finite second moment, and $U$ is sufficient for $\theta$ . Define the Rao–Blackwellized estimator $T^*=E(T\mid U)$ . Which statement about bias, variance, and mean squared error is necessarily correct?

T^*

is unbiased even when

T

is biased, and its variance cannot exceed that of

T

T^*

has the same bias as

T

and strictly smaller MSE whenever

T

is not a constant

T^*

has the same bias as

T

and no larger MSE, with equality if and only if

T=T^*

almost surely

T^*

has smaller absolute bias than

T

, but its variance can increase after conditioning

← Back to quizzes

Statistics Graduate Level Quiz

Statistics Graduate Level Quiz: Bias Variance And Mse

Practice Bias Variance And Mse in Statistics Graduate Level with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Bias Variance And Mse, giving you a quick way to practice the rules, question types, and explanations that matter most for Statistics Graduate Level.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

$T^*$ is unbiased even when $T$ is biased, and its variance cannot exceed that of $T$
$T^*$ has the same bias as $T$ and strictly smaller MSE whenever $T$ is not a constant
$T^*$ has the same bias as $T$ and no larger MSE, with equality if and only if $T=T^*$ almost surely (correct answer)
$T^*$ has smaller absolute bias than $T$ , but its variance can increase after conditioning

Explanation: Whenever you encounter a Rao–Blackwell question, anchor your thinking to two foundational tools: the law of total expectation and the law of total variance. These two identities fully determine what conditioning on a sufficient statistic does to bias, variance, and MSE. The law of total expectation gives

E(T^*) = E[E(T \mid U)] = E(T)

, so

T^*

and

T

share exactly the same bias — whatever it is. The law of total variance gives

\text{Var}(T) = \text{Var}(T^*) + E[\text{Var}(T \mid U)]

, meaning

\text{Var}(T^*) = \text{Var}(T) - E[\text{Var}(T \mid U)] \leq \text{Var}(T)

. Since bias is preserved and variance can only decrease, MSE satisfies

\text{MSE}(T^*) \leq \text{MSE}(T)

. Equality holds precisely when

E[\text{Var}(T \mid U)] = 0

, which happens if and only if

T = T^*

almost surely. This is exactly what C states, making it the correct answer. A is wrong because bias is preserved by conditioning —

T^*

is unbiased only if

T

already is. Claiming

T^*

corrects bias is a common misconception. B is wrong because it says MSE is strictly smaller whenever

T

is not a constant. In fact, strict improvement requires

T \neq T^*

almost surely — a deterministic (constant)

T

could still equal

T^*

. D is wrong on both counts: the variance of

T^*

never exceeds that of

T

, and absolute bias is not necessarily reduced (it is simply preserved). Your study tip: memorize that Rao–Blackwell preserves bias and never increases variance, with strict variance reduction if and only if

T \neq T^*

a.s. These two facts together answer almost every Rao–Blackwell multiple-choice question.

Question 2

In a fixed-design regression model, an ordinary least squares estimator satisfies $E(\widehat\beta)=\beta$ and $\operatorname{Var}(\widehat\beta)=v$ . A scalar ridge-type estimator is $\widetilde\beta_k=k\widehat\beta$ for a fixed $0<k<1$ .

For which condition does $\widetilde\beta_k$ have strictly smaller mean squared error than $\widehat\beta$ at the specified value of $\beta$ ?

$\beta^2<[(1+k)/(1-k)]v$ , because the variance reduction exceeds squared bias (correct answer)
$\beta^2<[(1-k)/(1+k)]v$ , because the shrinkage factor reduces both terms
$\beta^2>[(1+k)/(1-k)]v$ , because ridge improves most for large coefficients
$\beta^2>[(1-k)/(1+k)]v$ , because squared bias vanishes beyond this threshold

Explanation: When comparing two estimators, the key tool is the bias-variance decomposition of MSE:

\text{MSE}(\tilde\beta_k) = \text{Bias}^2 + \text{Var}

. For

\widetilde\beta_k = k\widehat\beta

, since

E(\widehat\beta) = \beta

, the bias is

E(k\widehat\beta) - \beta = (k-1)\beta

, so

\text{Bias}^2 = (1-k)^2\beta^2

. The variance is

\text{Var}(k\widehat\beta) = k^2 v

. Thus

\text{MSE}(\widetilde\beta_k) = (1-k)^2\beta^2 + k^2 v

, compared to

\text{MSE}(\widehat\beta) = v

(since OLS is unbiased). You want

\text{MSE}(\widetilde\beta_k) < v

, which means

(1-k)^2\beta^2 + k^2 v < v

. Rearranging:

(1-k)^2\beta^2 < v(1-k^2) = v(1-k)(1+k)

. Dividing both sides by

(1-k) > 0

gives

(1-k)\beta^2 < v(1+k)

, and finally

\beta^2 < \frac{(1+k)}{(1-k)}v

. This confirms answer A is correct — shrinkage wins when the true coefficient is small relative to the variance, because the variance reduction outweighs the introduced bias. Answer B flips the ratio to

(1-k)/(1+k)

, which is strictly less than 1 for

0 < k < 1

, making the condition far too restrictive — this comes from an algebra error when dividing by

(1+k)

instead of

(1-k)

. Answers C and D reverse the inequality direction entirely, claiming ridge improves for large

\beta

, which contradicts the intuition: shrinkage hurts most when the true signal is large, because bias grows while variance savings remain bounded. A useful pattern: whenever you see a shrinkage estimator, immediately write out

\text{MSE} = \text{Bias}^2 + \text{Var}

and solve the inequality algebraically — don't rely on intuition alone to get the exact threshold ratio.

Question 3

A two-arm experiment will enroll $40$ independent subjects. The treatment and control outcomes have variances $9$ and $1$ , respectively. The treatment measurement has a fixed additive calibration bias $d$ , while the control measurement is unbiased. The difference in sample means estimates the true treatment effect. Let $n_T+n_C=40$ .

Ignoring integer restrictions initially, which allocation minimizes the mean squared error of the estimated treatment effect?

$n_T=20$ and $n_C=20$ , because equal randomization removes allocation-induced bias
$n_T=36$ and $n_C=4$ , because allocation should be proportional to outcome variances
$n_T=10$ and $n_C=30$ , because more subjects should receive the unbiased measurement
$n_T=30$ and $n_C=10$ , because the fixed bias does not affect variance-optimal allocation (correct answer)

Explanation: When you see a question mixing bias and variance in an estimation problem, your instinct should be to write out the mean squared error (MSE) explicitly:

\text{MSE} = \text{Variance} + \text{Bias}^2

. Here, the estimator is

\hat{\Delta} = \bar{Y}_T - \bar{Y}_C

. The bias equals

d

(fixed, regardless of allocation), so

\text{Bias}^2 = d^2

is a constant that plays no role in optimization. The variance is:

\text{Var}(\hat{\Delta}) = \frac{\sigma_T^2}{n_T} + \frac{\sigma_C^2}{n_C} = \frac{9}{n_T} + \frac{1}{n_C}

To minimize this subject to

n_T + n_C = 40

, apply the Neyman allocation formula:

n_T \propto \sigma_T

and

n_C \propto \sigma_C

. Since

\sigma_T = 3

and

\sigma_C = 1

, the optimal split is

n_T : n_C = 3 : 1

, giving

n_T = 30

and

n_C = 10

— confirming D. A is wrong because equal allocation (20/20) ignores the unequal variances. Equal randomization controls confounding, but it does not minimize MSE when variances differ. B confuses the allocation rule — Neyman allocation is proportional to the standard deviation, not the variance. Proportional to variances would give 36/4, which is too extreme. C inverts the logic entirely; you should oversample the noisier arm (treatment,

\sigma=3

), not the quieter one. The key study tip: bias terms that are constant with respect to the design parameter vanish during optimization. Only the variance component drives the optimal allocation. Don't let the word "bias" in the problem distract you from a clean variance-minimization exercise.

Question 4

Suppose $\theta\sim N(0,\tau^2)$ and, conditionally on $\theta$ , $X\mid\theta\sim N(\theta,\sigma^2)$ . Consider estimators $\delta_c(X)=cX$ , where $c$ is a constant.

Which value of $c$ minimizes the integrated mean squared error $E[(\delta_c(X)-\theta)^2]$ , where expectation is over both $\theta$ and $X$ ?

$c=1$ , because conditional unbiasedness minimizes integrated squared error
$c=\tau/(\tau+\sigma)$ , balancing the two component standard deviations
$c=\sigma^2/(\tau^2+\sigma^2)$ , weighting observations by their noise variance
$c=\tau^2/(\tau^2+\sigma^2)$ , balancing prior variation against sampling noise (correct answer)

Explanation: When you see a question involving a prior distribution on a parameter and a conditional likelihood, you're in Bayesian estimation territory. The key move is to compute the integrated (joint) MSE by expanding it using the law of total expectation. Write the loss as

E[(\delta_c(X)-\theta)^2] = E[(cX - \theta)^2]

. Since

X = \theta + \epsilon

where

\epsilon \sim N(0,\sigma^2)

independently of

\theta

, substitute:

cX - \theta = (c-1)\theta + c\epsilon

. Because

\theta

and

\epsilon

are independent with means zero, the MSE becomes

(c-1)^2\tau^2 + c^2\sigma^2

. Minimize over

c

by differentiating and setting equal to zero:

2(c-1)\tau^2 + 2c\sigma^2 = 0

, giving

c(\tau^2 + \sigma^2) = \tau^2

, so

c = \tau^2/(\tau^2+\sigma^2)

. This is answer D, and it's exactly the Bayes estimator under squared error loss — the posterior mean

E[\theta \mid X]

is linear in

X

with this shrinkage coefficient. Choice A sets

c=1

, which is the MLE

\delta(X)=X

. It's conditionally unbiased for

\theta

, but ignores the prior, so it over-fits to noisy observations and inflates integrated MSE. Choice B,

c = \tau/(\tau+\sigma)

, looks plausible but mixes standard deviations rather than variances — the correct weighting involves squared quantities. Choice C flips the numerator and denominator, weighting by noise variance

\sigma^2

instead of prior variance

\tau^2

; this would shrink more when the signal is stronger, which is backwards. As a study tip: whenever you minimize an integrated risk over a linear estimator, expand the squared error algebraically using independence, then differentiate. The answer will always involve ratios of variances, not standard deviations.

Question 5

Let $X_1,\ldots,X_n$ be independent normal observations with unknown mean $\mu$ and variance $\sigma^2$ . Consider estimators of the form $T_c=c\sum_{i=1}^n(X_i-\bar X)^2$ , where $c$ is a constant that cannot depend on the unknown parameters.

Which value of $c$ minimizes $MSE(T_c)$ as an estimator of $\sigma^2$ ?

$c=1/(n-1)$ , which makes the estimator exactly unbiased
$c=1/n$ , which uses the maximum-likelihood variance divisor
$c=1/(n+1)$ , which accepts bias to reduce total MSE (correct answer)
$c=1/(n+2)$ , which applies two additional degrees of freedom

Explanation: When estimating a parameter, minimizing MSE requires balancing bias and variance — and sometimes accepting bias actually pays off. Here,

T_c = c\sum(X_i - \bar{X})^2

. Since

\sum(X_i - \bar{X})^2/\sigma^2 \sim \chi^2_{n-1}

, we know

E\left[\sum(X_i-\bar{X})^2\right] = (n-1)\sigma^2

, so

E[T_c] = c(n-1)\sigma^2

. The bias is

[c(n-1)-1]\sigma^2

, and it can be shown that

\text{Var}(T_c) = 2c^2(n-1)\sigma^4

. Therefore:

\text{MSE}(T_c) = \text{Bias}^2 + \text{Var} = [c(n-1)-1]^2\sigma^4 + 2c^2(n-1)\sigma^4

Taking

d(\text{MSE})/dc = 0

and solving yields

c = \frac{1}{n+1}

, confirming C is correct. This value deliberately introduces a small downward bias, but the reduction in variance more than compensates, lowering total MSE. Choice A (

c = 1/(n-1)

) gives the unbiased estimator

S^2

, but zero bias doesn't mean minimum MSE — variance still inflates the MSE unnecessarily. Choice B (

c = 1/n

) is the MLE, which also fails to minimize MSE; the MLE optimizes likelihood, not squared-error loss. Choice D (

c = 1/(n+2)

) goes too far — overshooting the optimal bias-variance tradeoff actually increases MSE relative to

1/(n+1)

. Study tip: On MSE-minimization problems, always write out

\text{MSE} = \text{Bias}^2 + \text{Variance}

explicitly, differentiate with respect to your free parameter, and solve. Never assume the unbiased or MLE estimator automatically minimizes MSE — that's the most common trap in this topic area.

Question 6

Suppose $\bar X\sim N(\mu,\sigma^2/n)$ , where $\sigma^2$ is known. Two estimators of $\mu^2$ are $T_1=\bar X^2$ and $T_2=\bar X^2-\sigma^2/n$ .

Which statement correctly compares the mean squared errors of these estimators?

$MSE(T_1)-MSE(T_2)=\sigma^4/n^2$ , so the unbiased correction lowers MSE (correct answer)
$MSE(T_2)-MSE(T_1)=\sigma^4/n^2$ , so the correction raises variance
$MSE(T_1)=MSE(T_2)$ , because subtracting a constant leaves variance unchanged
$MSE(T_1)-MSE(T_2)=2\sigma^4/n^2$ , because normal fourth moments double the bias

Explanation: When comparing estimators, remember that

MSE = \text{Bias}^2 + \text{Variance}

. The key here is computing bias and variance for each estimator separately, then comparing. Since

\bar{X} \sim N(\mu, \sigma^2/n)

, we have

E[\bar{X}^2] = \text{Var}(\bar{X}) + (E[\bar{X}])^2 = \sigma^2/n + \mu^2

. So

T_1 = \bar{X}^2

has bias

\sigma^2/n

, giving

\text{Bias}^2(T_1) = \sigma^4/n^2

. Meanwhile

T_2 = \bar{X}^2 - \sigma^2/n

is unbiased for

\mu^2

, so

\text{Bias}^2(T_2) = 0

. Both estimators share the same variance term

\text{Var}(\bar{X}^2)

, since subtracting the constant

\sigma^2/n

doesn't affect variance. Therefore

MSE(T_1) - MSE(T_2) = \sigma^4/n^2

, confirming A: the bias correction strictly lowers MSE. Choice B reverses the subtraction, claiming the correction raises MSE — but since

T_2

eliminates the positive bias-squared term without touching variance, it must have lower MSE, not higher. Choice C is tempting because subtracting a constant truly does leave variance unchanged — but MSE includes bias, not just variance, and that's exactly where the difference lives. C confuses

\text{Var}(T)

with

MSE(T)

. Choice D invokes a spurious "doubling" from normal fourth moments; while higher moments of the normal distribution do appear in

\text{Var}(\bar{X}^2)

, they cancel identically between

T_1

and

T_2

, contributing nothing to their difference. The study tip: whenever you see an estimator formed by adding or subtracting a constant, remember that only the bias changes — variance is unaffected. Always decompose MSE into its two components before drawing conclusions.

Question 7

A regular model based on $n=9$ independent observations has Fisher information $1$ per observation for the scalar parameter $\theta$ . An estimator $T$ has differentiable bias function $b(\theta)=-\theta/4$ .

At $\theta=2$ , what lower bound on $MSE(T)$ follows from the biased Cramér–Rao inequality?

$1/16$ , obtained from the variance lower bound without adding squared bias
$1/4$ , obtained by retaining squared bias but omitting sampling variance
$5/16$ , obtained by combining the biased variance bound and squared bias (correct answer)
$13/36$ , obtained by applying the unbiased information bound before adding bias

Explanation: Whenever you see a biased estimator paired with the Cramér–Rao inequality, remember that the bound applies to variance, not MSE directly — but MSE includes both variance and squared bias, so you must combine them carefully. The biased Cramér–Rao lower bound states that for an estimator with bias

b(\theta)

, its variance satisfies:

\text{Var}(T) \geq \frac{(1 + b'(\theta))^2}{nI(\theta)}

Here,

b(\theta) = -\theta/4

, so

b'(\theta) = -1/4

. With

n = 9

and

I(\theta) = 1

\text{Var}(T) \geq \frac{(1 - 1/4)^2}{9 \cdot 1} = \frac{(3/4)^2}{9} = \frac{9/16}{9} = \frac{1}{16}

Now add the squared bias at

\theta = 2

b(2) = -2/4 = -1/2

, so

b(2)^2 = 1/4

. The full MSE lower bound is:

\text{MSE}(T) \geq \frac{1}{16} + \frac{1}{4} = \frac{1}{16} + \frac{4}{16} = \frac{5}{16}

This confirms C is correct. Choice A stops at the variance bound

(1/16)

and forgets to add the squared bias — a classic incomplete application of MSE decomposition. Choice B keeps the squared bias

(1/4)

but discards the variance contribution entirely, which is equally incomplete. Choice D uses

nI(\theta) = 9

in the denominator without the bias-adjustment factor

(1+b')^2

, as if the estimator were unbiased, then awkwardly adds bias — mixing two incompatible frameworks. Study tip: Always write

\text{MSE} = \text{Var} + \text{Bias}^2

first, then bound each piece separately. The biased CRB only touches the variance term — never skip the squared bias when the question asks about MSE.

Question 8

Two estimators of $\theta$ satisfy $E(T_1)-\theta=1$ , $E(T_2)-\theta=-1$ , $\operatorname{Var}(T_1)=4$ , $\operatorname{Var}(T_2)=1$ , and $\operatorname{Cov}(T_1,T_2)=1$ . For $T_a=aT_1+(1-a)T_2$ , where $a$ may be any real number, which value of $a$ minimizes the mean squared error of $T_a$ ?

$a=0$ , because the second estimator has the smaller variance
$a=2/7$ , after accounting for both covariance and squared bias (correct answer)
$a=1/3$ , after treating the two estimators as independent
$a=1/2$ , because this makes the combined estimator unbiased

Explanation: When minimizing mean squared error (MSE) for a linear combination of biased estimators, you must account for all three components: variance of each estimator, their covariance, and the squared bias of the combined estimator. Start by computing the bias of

T_a

. Since

E(T_1) = \theta + 1

and

E(T_2) = \theta - 1

, the bias of

T_a = aT_1 + (1-a)T_2

a(1) + (1-a)(-1) = 2a - 1

. Next, the variance of

T_a

a^2\operatorname{Var}(T_1) + (1-a)^2\operatorname{Var}(T_2) + 2a(1-a)\operatorname{Cov}(T_1,T_2) = 4a^2 + (1-a)^2 + 2a(1-a)

, which simplifies to

3a^2 - 2a + 1 + 2a(1-a) = 3a^2 + 1 - 2a + 2a - 2a^2 = a^2 - 2a + 1 + 2a + 2 - 2a = ...

Carefully expanding:

4a^2 + 1 - 2a + a^2 + 2a - 2a^2 = 3a^2 + 1

. So

\operatorname{MSE}(T_a) = 3a^2 + 1 + (2a-1)^2 = 3a^2 + 1 + 4a^2 - 4a + 1 = 7a^2 - 4a + 2

. Differentiating and setting equal to zero:

14a - 4 = 0 \Rightarrow a = 2/7

. This confirms answer B is correct. Choice A ignores that

T_2

's smaller variance doesn't automatically make

a=0

optimal once biases differ. Choice C (

a = 1/3

) drops the covariance term, treating the estimators as independent and arriving at the wrong minimum. Choice D (

a = 1/2

) achieves unbiasedness but ignores that MSE includes variance — unbiasedness alone is not optimality. Study tip: Always write out MSE = Variance + Bias² fully before differentiating. Forgetting the covariance term is the most common error on linear-combination optimization problems.

Question 9

An estimator $T_n$ has $\operatorname{Bias}(T_n)=2/n+o(n^{-1})$ and $\operatorname{Var}(T_n)=1/n$ . A bootstrap bias-corrected estimator $T_n^{BC}$ has $\operatorname{Bias}(T_n^{BC})=o(n^{-1})$ and $\operatorname{Var}(T_n^{BC})=1/n+6/n^2+o(n^{-2})$ .

Which comparison is correct to order $n^{-2}$ ?

Bias correction lowers MSE by $2/n^2$ because its leading bias is removed
Bias correction raises MSE by $2/n^2$ because variance inflation exceeds the saved squared bias (correct answer)
The estimators have equal MSE through order $n^{-2}$ because both variances begin with $1/n$
Bias correction raises MSE by $6/n^2$ because the original estimator's bias is asymptotically negligible

Explanation: When comparing estimators via MSE, remember that

\text{MSE} = \text{Bias}^2 + \text{Variance}

. Bootstrap bias correction reduces squared bias but inflates variance — and you must track both effects carefully to the same order in

n

. For the original estimator

T_n

\text{Bias}^2 = (2/n)^2 = 4/n^2

and

\text{Var} = 1/n

, so

\text{MSE}(T_n) = 1/n + 4/n^2 + o(n^{-2})

. For the bias-corrected estimator

T_n^{BC}

\text{Bias}^2 = o(n^{-2})

(negligible) and

\text{Var} = 1/n + 6/n^2

, so

\text{MSE}(T_n^{BC}) = 1/n + 6/n^2 + o(n^{-2})

. The difference is

\text{MSE}(T_n^{BC}) - \text{MSE}(T_n) = 6/n^2 - 4/n^2 = +2/n^2

. Bias correction raises MSE by

2/n^2

because the variance inflation of

6/n^2

exceeds the recovered squared bias of

4/n^2

. That confirms B. Choice A gets the direction wrong — it claims MSE decreases, but the variance penalty outweighs the bias savings. Choice C is tempting because both estimators share the

1/n

variance leading term, but the

n^{-2}

terms differ critically — stopping at the leading variance term misses the entire comparison. Choice D correctly identifies that variance inflation drives the MSE increase, but assigns the wrong magnitude (

6/n^2

instead of the net

2/n^2

), confusing gross variance increase with net MSE change. The strategy: always compute

\text{MSE} = \text{Bias}^2 + \text{Variance}

term by term to the required order before drawing conclusions — never evaluate bias and variance in isolation.

Question 10

For an estimator sequence $T_n$ of $\theta$ , suppose $\operatorname{Bias}(T_n)=n^{-1/4}+o(n^{-1/4})$ and $\operatorname{Var}(T_n)=2n^{-1/2}+o(n^{-1/2})$ . Which conclusion follows?

$T_n$ is inconsistent because its bias decreases more slowly than the usual root-sample-size rate
$T_n$ is consistent, and $n^{1/2}MSE(T_n)\to 3$ (correct answer)
$T_n$ is consistent, and $n^{1/2}MSE(T_n)\to 2$
$T_n$ is root-sample-size consistent because $nMSE(T_n)$ has a finite limit

Explanation: When analyzing MSE asymptotics, your first instinct should be to decompose:

\text{MSE}(T_n) = \text{Bias}^2(T_n) + \text{Var}(T_n)

, then identify which term dominates. Here,

\text{Bias}^2(T_n) = (n^{-1/4} + o(n^{-1/4}))^2 = n^{-1/2} + o(n^{-1/2})

, and

\text{Var}(T_n) = 2n^{-1/2} + o(n^{-1/2})

. Adding them gives

\text{MSE}(T_n) = 3n^{-1/2} + o(n^{-1/2})

. Multiplying by

n^{1/2}

n^{1/2}\text{MSE}(T_n) \to 3

. Since

\text{MSE}(T_n) \to 0

, the estimator is consistent. This confirms B. A is the most tempting trap. It's true that bias shrinks at rate

n^{-1/4}

, slower than the familiar

n^{-1/2}

. But consistency only requires

\text{MSE}(T_n) \to 0

, which holds here since both the squared bias and variance vanish. Slow convergence ≠ inconsistency. C is a calculation error — students who forget to square the bias before adding it to the variance would see only the variance term contributing, getting a limit of 2 instead of 3. Always square the bias first. D would require

n \cdot \text{MSE}(T_n) \to L < \infty

, i.e.,

\text{MSE} = O(n^{-1})

. But here

\text{MSE} \sim 3n^{-1/2}

, so

n \cdot \text{MSE} \sim 3n^{1/2} \to \infty

. Root-

n

consistency fails. Study tip: Always square the bias before computing MSE limits — it changes the convergence rate and can flip which term dominates. This is the single most common arithmetic error in asymptotic MSE problems.