Unbiased Estimation & UMVUE - Statistics Graduate Level

1. Let $X_1,\ldots,X_n$ be independent with density $f_\theta(x)=1/\theta$ for $0<x<\theta$, where $\theta>0$. Write $M=\max_i X_i$. Which function of $M$ is the UMVUE of $\theta^2$?
   Answer: $\frac{n+2}{n}M^2$ When you see a question like this, think: sufficient statistic → Rao-Blackwell → UMVUE. For the Uniform$(0,\theta)$ family, $M = \max_i X_i$ is a complete sufficient statistic, so any unbiased estimator that is a function of $M$ is automatically the UMVUE. Your job is to find which function of $M$ is unbiased for $\theta^2$. Start by deriving the distribution of $M$. Its CDF is $F_M(m) = (m/\theta)^n$, giving density $f_M(m) = nm^{n-1}/\theta^n$ for $0 < m < \theta$. Now compute $E[M^2]$: $E[M^2] = \int_0^\theta m^2 \cdot \frac{nm^{n-1}}{\theta^n}\,dm = \frac{n}{\theta^n}\cdot\frac{\theta^{n+2}}{n+2} = \frac{n}{n+2}\,\theta^2$ So $E[M^2] = \frac{n}{n+2}\theta^2$, which means to get an unbiased estimator of $\theta^2$ you need to multiply $M^2$ by $\frac{n+2}{n}$. That confirms B is correct. A uses the factor $\frac{n+1}{n}$, which corresponds to making $\frac{n+1}{n}M$ unbiased for $\theta$ (since $E[M] = \frac{n}{n+1}\theta$) — a different problem entirely. C inverts the correction, giving $\frac{n}{n+2}M^2$, which has expectation $\frac{n^2}{(n+2)^2}\theta^2 \neq \theta^2$. D squares the UMVUE of $\theta$, yielding $\frac{(n+1)^2}{n^2}M^2$, which is biased for $\theta^2$ since squaring an unbiased estimator introduces bias. Study tip: Always compute $E[M^k]$ explicitly from the order-statistic density — confusing the unbiasing constants for different powers of $\theta$ is the most common trap here.
2. Which estimator is the UMVUE of $\beta_j^2$?
   Answer: $\widehat\beta_j^2-\sigma^2v_j$ When a question asks for the UMVUE of a nonlinear function of parameters, your instinct should be the Rao-Blackwell/Lehmann-Scheffé approach: find an unbiased estimator that is a function of the complete sufficient statistic. In the normal linear model with known $\sigma^2$, the complete sufficient statistic is $\widehat{\beta}$, so any unbiased function of $\widehat{\beta}$ is automatically the UMVUE. The key insight is that $\widehat{\beta}_j \sim N(\beta_j, \sigma^2 v_j)$, which means $E[\widehat{\beta}_j^2] = \text{Var}(\widehat{\beta}_j) + (E[\widehat{\beta}_j])^2 = \sigma^2 v_j + \beta_j^2$. Therefore $E[\widehat{\beta}_j^2 - \sigma^2 v_j] = \beta_j^2$, making answer A an unbiased estimator. Since it is a function of the complete sufficient statistic $\widehat{\beta}$, by Lehmann-Scheffé it is the UMVUE. B adds $\sigma^2 v_j$ instead of subtracting it, giving $E[\widehat{\beta}_j^2 + \sigma^2 v_j] = \beta_j^2 + 2\sigma^2 v_j \neq \beta_j^2$. This reflects a sign error in the bias-correction step. C subtracts $\sigma v_j$ (note: $\sigma$, not $\sigma^2$), which is dimensionally inconsistent and leaves a residual bias. D divides $\sigma^2$ by $v_j$ rather than multiplying, which has no grounding in the variance formula for $\widehat{\beta}_j$ and does not yield an unbiased estimator. A useful strategy: whenever you need the UMVUE of $g(\theta)$ for a curved function, compute $E[T^2]$ for your sufficient statistic $T$, identify the bias term explicitly, and subtract it. The bias of $\widehat{\beta}_j^2$ is always $+\sigma^2 v_j$, so the correction always goes in the negative direction.
3. Let $U$ be an unbiased estimator with finite variance, and let $T$ be sufficient. Suppose $P_\theta\{\operatorname{Var}_\theta(U\mid T)>0\}>0$ for every $\theta$. What follows about the Rao–Blackwell estimator $E_\theta(U\mid T)$?
   Answer: It has strictly smaller variance than $U$ for every $\theta$. Whenever you encounter a Rao–Blackwell question, your anchor should be the law of total variance: for any random variables $X$ and $Y$, $\operatorname{Var}(X) = E[\operatorname{Var}(X \mid Y)] + \operatorname{Var}(E[X \mid Y]).$ This identity is the engine behind everything here. Let $\tilde{U} = E_\theta(U \mid T)$ be the Rao–Blackwell estimator. Applying the law of total variance to $U$ conditioned on $T$: $\operatorname{Var}_\theta(U) = E_\theta[\operatorname{Var}_\theta(U \mid T)] + \operatorname{Var}_\theta(\tilde{U}).$ Rearranging, $\operatorname{Var}_\theta(\tilde{U}) = \operatorname{Var}_\theta(U) - E_\theta[\operatorname{Var}_\theta(U \mid T)].$ The condition $P_\theta\{\operatorname{Var}_\theta(U \mid T) > 0\} > 0$ guarantees that $E_\theta[\operatorname{Var}_\theta(U \mid T)] > 0$ for every $\theta$, so $\operatorname{Var}_\theta(\tilde{U}) < \operatorname{Var}_\theta(U)$ strictly, confirming D. A is wrong because without completeness of $T$, you cannot guarantee uniqueness or optimality — Rao–Blackwell improves variance, but UMVUE status requires the Lehmann–Scheffé theorem and a complete sufficient statistic. B confuses unbiasedness with equal variance; both estimators are indeed unbiased (since $E_\theta[\tilde{U}] = E_\theta[U]$ by the tower property), but unbiasedness says nothing about variance equality. C is doubly wrong: Rao–Blackwellization preserves unbiasedness so bias does not change, and as shown, variance strictly decreases, not increases. Study tip: Memorize the law of total variance as your go-to tool for Rao–Blackwell problems — the improvement in variance is always exactly $E_\theta[\operatorname{Var}_\theta(U \mid T)]$, which is positive precisely when $U$ is not already a function of $T$.
4. Let $X_1,\ldots,X_n$ be independent Poisson random variables with common mean $\lambda$, where $n\ge 2$. The estimator $U=\mathbf{1}\{X_1=0\}$ is unbiased for $e^{-\lambda}$. If $T=\sum_{i=1}^n X_i$, which estimator is the Rao–Blackwell improvement of $U$ and hence the UMVUE of $e^{-\lambda}$?
   Answer: $\left(1-\frac{1}{n}\right)^T$ When you see a question involving unbiased estimation and a sufficient statistic, your immediate instinct should be to apply the Rao–Blackwell theorem: condition the unbiased estimator $U$ on the complete sufficient statistic $T$ to obtain the UMVUE. Here, $T = \sum_{i=1}^n X_i \sim \text{Poisson}(n\lambda)$ is complete sufficient for $\lambda$. You need to compute $E[U \mid T] = E[\mathbf{1}\{X_1 = 0\} \mid T = t]$, which equals $P(X_1 = 0 \mid T = t)$. Since $T = t$ distributes the total count among $n$ Poisson variables, this conditional distribution is Binomial$(t, 1/n)$, so: $P(X_1 = 0 \mid T = t) = \left(1 - \frac{1}{n}\right)^t$ Replacing $t$ with $T$ gives the Rao–Blackwell estimator $\left(1-\frac{1}{n}\right)^T$, which is answer A. Since $T$ is complete sufficient, this is automatically the UMVUE. Answer B, $\left(1-\frac{1}{n}\right)^{T/n}$, incorrectly divides the exponent by $n$ — a tempting but unjustified modification. Answer C, $\mathbf{1}\{T=0\}$, is unbiased for $e^{-n\lambda}$, not $e^{-\lambda}$, so it estimates the wrong quantity entirely. Answer D, $\exp(-T/n)$, is a natural plug-in estimator (replacing $\lambda$ with $\bar{X}$), but it is biased — $E[e^{-\bar{X}}] \neq e^{-\lambda}$ by Jensen's inequality. Your study tip: always derive the conditional probability explicitly rather than guessing the form. The binomial conditional distribution of $X_1 \mid T$ is a standard result you should have memorized for Poisson problems.
5. Suppose $X_1,\ldots,X_n$ are independent Bernoulli random variables with success probability $p$, where $0<p<1$ and $n$ is fixed. Which target function admits an unbiased estimator based on this sample?
   Answer: $p^n$ When working with Bernoulli samples, a key question is: which functions of $p$ are estimable? A function $g(p)$ admits an unbiased estimator if and only if it can be expressed as a polynomial in $p$ of degree at most $n$ — this follows from the fact that any estimator's expectation must be a polynomial in $p$ of degree $\leq n$ (since the sufficient statistic $S = \sum X_i \sim \text{Binomial}(n,p)$). Consider choice A: $g(p) = p^n$. The estimator $T = X_1 X_2 \cdots X_n$ — the product of all observations — is unbiased, since $E[X_1 \cdots X_n] = p^n$ by independence. This is a degree-$n$ polynomial in $p$, so it's estimable. A is correct. Choice B, $p^{n+1}$, requires a degree-$(n+1)$ polynomial, which exceeds what any estimator based on $n$ Bernoulli observations can reproduce in expectation. No unbiased estimator exists. Choice C, $\frac{1}{p}$, is not a polynomial in $p$ at all — it diverges as $p \to 0$, while any estimator's expectation must be a polynomial (hence bounded near zero). This rules out unbiasedness entirely. Choice D, $\log\!\left(\frac{p}{1-p}\right)$, is a transcendental function, not a polynomial, so it similarly cannot equal the expectation of any estimator for all $p \in (0,1)$. Study tip: For estimability questions with exponential family models, always check whether the target function is a polynomial in $p$ of degree $\leq n$. If it's transcendental or exceeds degree $n$, no unbiased estimator can exist.
6. Let $X_1,\ldots,X_n$ be independent exponential random variables with rate $\theta>0$ and density $f_\theta(x)=\theta e^{-\theta x}$ for $x>0$. Assume $n>1$ and set $S=\sum_{i=1}^n X_i$. Which estimator is the UMVUE of the rate $\theta$?
   Answer: $\frac{n-1}{S}$ When you see a question asking for a UMVUE, your roadmap is: find a complete sufficient statistic, then find an unbiased estimator that's a function of it. By the Lehmann-Scheffé theorem, that estimator is the UMVUE. For exponential data with rate $\theta$, the sufficient statistic is $S = \sum_{i=1}^n X_i$, which follows a Gamma$(n, \theta)$ distribution — meaning its density involves $\theta^n e^{-\theta s}$, confirming completeness. Now you need an unbiased estimator of $\theta$ based on $S$. Since $S \sim \text{Gamma}(n, \theta)$, its mean is $E[S] = n/\theta$, so $E[1/S] = \theta/(n-1)$ (using the inverse-gamma expectation: $E[1/S] = \theta/(n-1)$ for $n > 1$). Therefore $E\!\left[\frac{n-1}{S}\right] = \theta$, making B an unbiased estimator of $\theta$ that depends only on the complete sufficient statistic $S$. By Lehmann-Scheffé, it is the UMVUE. Choice A, $\frac{n}{S}$, is the MLE of $\theta$ (since $\hat{\theta}_{MLE} = n/S$), but it is biased: $E[n/S] = n \cdot \theta/(n-1) \neq \theta$. It overestimates $\theta$ on average. Choice C, $\frac{S}{n-1}$, estimates $1/\theta$ (the mean), not the rate — it's solving the wrong problem entirely. Choice D, $\frac{n+1}{S}$, overcorrects in the wrong direction and has no natural derivation here. A key study tip: the MLE is rarely unbiased — always verify unbiasedness separately, especially for rate/precision parameters where the bias correction flips the sign of the adjustment (you subtract from $n$, not add).
7. Which statement correctly describes the role of completeness in this model?
   Answer: $T$ is not complete because $E_\theta\{H(T)\}=0$ at both parameter values although $H(T)$ is not almost surely zero. Completeness is a property of a sufficient statistic that goes beyond sufficiency alone. A statistic $T$ is complete for a family indexed by $\theta$ if the only measurable function $h(T)$ satisfying $E_\theta[h(T)] = 0$ for all $\theta$ in the parameter space is $h(T) = 0$ almost surely. When the parameter space is unusually small, this condition can fail in surprising ways. Here, consider $H(T) = T^2 - 2$. Under $\theta = 1$, we have $X \sim N(1,1)$, so $E_1[X^2] = \text{Var}(X) + (E_1[X])^2 = 1 + 1 = 2$, giving $E_1[H(T)] = 2 - 2 = 0$. Under $\theta = -1$, symmetry gives $E_{-1}[X^2] = 1 + 1 = 2$ as well, so $E_{-1}[H(T)] = 0$. Since $H(T)$ is zero in expectation at both parameter values yet is not almost surely zero (it varies with $X$), completeness fails. This confirms C is correct. A is wrong because sufficiency is unaffected by the size of the parameter space — $T = X$ still carries all the information about $\theta$, which is what sufficiency measures. B is wrong because completeness of a normal family holds when the parameter space contains an open interval; restricting $\theta$ to just two points breaks this, as the example demonstrates. D is wrong because nonzero variance of $H(T)$ is irrelevant — completeness requires checking whether the expectation condition forces $h = 0$ a.s., not whether $h$ has spread. As a study tip: whenever a question restricts the parameter space to a finite or otherwise thin set, immediately suspect that completeness may fail even if sufficiency is intact.
8. Among estimators of the form $U+cH$, which one has the smallest variance, and what does this imply about $U$?
   Answer: $U-\frac{1}{2}H$; therefore $U$ cannot be the UMVUE. Whenever you see a question about minimizing variance over a family of estimators, your instinct should be to treat the variance as a function of the free parameter and optimize it directly. Since $H$ is unbiased for zero, $U + cH$ is unbiased for $g(\theta)$ for any constant $c$. Now expand the variance: $\operatorname{Var}(U + cH) = \operatorname{Var}(U) + 2c\operatorname{Cov}(U,H) + c^2\operatorname{Var}(H)$ $= \operatorname{Var}(U) + 2c(2) + c^2(4) = \operatorname{Var}(U) + 4c + 4c^2$ To minimize, take the derivative with respect to $c$ and set it to zero: $\frac{d}{dc}(4c + 4c^2) = 4 + 8c = 0 \implies c = -\frac{1}{2}$ So the minimum-variance estimator in this family is $U - \frac{1}{2}H$, confirming answer D. The critical implication: if $U$ were the UMVUE, no unbiased estimator could have smaller variance than $U$. But $U - \frac{1}{2}H$ (with $c = -\frac{1}{2} \neq 0$) strictly beats $U$, so $U$ cannot be the UMVUE. A uses $c = 2$, which is not the minimizer — it's a common error from forgetting the factor of 2 in the derivative. B uses $c = +\frac{1}{2}$, getting the sign wrong; and even if the variance were reduced, claiming $U$ "cannot be UMVUE" would be correct, making this a sign-error trap. C finds the right magnitude $c = -2$ but misapplies the formula — and then draws the wrong conclusion about UMVUE status. The key study tip: any time a nonzero $c$ minimizes the variance, the original estimator $U$ is automatically disqualified as a UMVUE. The sign of the optimal $c$ comes directly from $-\frac{\operatorname{Cov}(U,H)}{\operatorname{Var}(H)}$, so memorize this ratio.
9. Which conclusion is justified from the stated information alone?
   Answer: $\delta(T)$ is unbiased and has variance no greater than that of $U$, but it need not be the UMVUE. Whenever you see a question combining sufficiency, conditioning, and optimality, you need to carefully separate what the Rao–Blackwell Theorem guarantees from what the Lehmann–Scheffé Theorem adds on top of it. The Rao–Blackwell Theorem tells you exactly two things: if $U$ is unbiased for $g(\theta)$ and $T$ is sufficient, then $\delta(T) = E_\theta(U \mid T)$ is (1) still unbiased, because $E_\theta[\delta(T)] = E_\theta[E_\theta(U \mid T)] = E_\theta[U] = g(\theta)$, and (2) has variance no greater than $\text{Var}(U)$, by the law of total variance: $\text{Var}(U) = \text{Var}(\delta(T)) + E[\text{Var}(U \mid T)] \geq \text{Var}(\delta(T))$. That's all sufficiency alone gives you — making A correct. To guarantee $\delta(T)$ is the UMVUE, you need the additional condition that $T$ is complete. Completeness ensures uniqueness among unbiased estimators of $T$, which is the content of Lehmann–Scheffé. Since the problem explicitly states completeness is not known, you cannot claim UMVUE status. B is wrong because sufficiency does not imply completeness — these are distinct properties, and conflating them is a classic trap. C is wrong for the same reason: Rao–Blackwell improves variance but does not guarantee uniform optimality without completeness. D is wrong because the tower property of conditional expectation preserves unbiasedness — conditioning on a sufficient statistic never destroys it. Your study tip: memorize the two-step ladder — Rao–Blackwell gives you improvement, Lehmann–Scheffé (requiring completeness) gives you uniqueness and optimality. Exam questions often test whether you confuse the two.
10. Let $X_1,\ldots,X_n$ be independent Poisson random variables with mean $\lambda$, where $n\ge 2$. The estimator $U=X_1X_2$ is unbiased for $\lambda^2$. If $T=\sum_{i=1}^n X_i$, which expression equals $E(U\mid T)$ and is therefore the UMVUE of $\lambda^2$?
    Answer: $\frac{T(T-1)}{n^2}$ Whenever you see a question involving unbiased estimators and sufficient statistics, think Rao-Blackwell: if you condition an unbiased estimator on a complete sufficient statistic, you get the UMVUE. Here, $T = \sum_{i=1}^n X_i$ is complete sufficient for $\lambda$ (since Poisson belongs to the exponential family), and $U = X_1 X_2$ is unbiased for $\lambda^2$, so $E(U \mid T)$ is the UMVUE. To compute $E(X_1 X_2 \mid T = t)$, use the fact that given $T = t$, the vector $(X_1, \ldots, X_n)$ follows a multinomial distribution with parameters $t$ and equal probabilities $1/n$. Therefore: $E(X_1 X_2 \mid T = t) = E[X_1(X_1-1) + X_1 \mid T=t]$ Wait — more directly: since $E(X_1^2 \mid T=t) = \text{Var}(X_1\mid T) + [E(X_1\mid T)]^2$, use the multinomial result that $E(X_i X_j \mid T = t) = \frac{t(t-1)}{n^2}$ for $i \neq j$. This follows because each pair draw is without-replacement-style: $P(X_1 \geq 1, X_2 \geq 1 \mid T = t) \propto t(t-1)$ selections across $n^2$ ordered slots. The result is $\frac{T(T-1)}{n^2}$, confirming C. A uses $n(n-1)$ in the denominator — a tempting combinatorial normalization, but incorrect; the denominator should be $n^2$ reflecting two independent draws each with probability $1/n$. B gives $T^2/n^2$, which would be correct only if $X_1$ and $X_2$ were independent given $T$, but they're negatively correlated conditionally. D adds an ad hoc correction term with no probabilistic justification. Your study tip: memorize that conditional on $T = t$ for i.i.d. Poissons, $E(X_i X_j \mid T) = \frac{T(T-1)}{n^2}$ for $i \neq j$ — this exact calculation appears frequently in UMVUE derivations.

Start this quiz