Multivariate Normal Distribution - Statistics Graduate Level

1. Let $X=(X_1,X_2,X_3)^{\mathsf T}$ have a multivariate normal distribution with mean zero and covariance matrix $\Sigma=\begin{pmatrix}2&1&0\\1&2&1\\0&1&2\end{pmatrix}.$ What is the conditional distribution of $X_2$ given $X_1+X_3=2$?
   Answer: $N(1,1)$, because the conditioning variable has variance $4$ and covariance $2$ with $X_2$. When you condition on a linear combination of jointly normal variables, the key tool is the bivariate normal conditional formula. Reframe the problem: let $Y = X_1 + X_3$. Since $(X_2, Y)$ is jointly normal (linear combinations preserve normality), you just need $\text{Var}(Y)$, $\text{Cov}(X_2, Y)$, and the conditional mean/variance formulas. First, compute $\text{Var}(Y) = \text{Var}(X_1) + \text{Var}(X_3) + 2\text{Cov}(X_1,X_3) = 2 + 2 + 2(0) = 4$. Next, $\text{Cov}(X_2, Y) = \text{Cov}(X_2, X_1) + \text{Cov}(X_2, X_3) = 1 + 1 = 2$. Both means are zero, so the conditional mean is $\mu_{2|Y} = 0 + \frac{2}{4}(2 - 0) = 1$. The conditional variance is $\text{Var}(X_2) - \frac{[\text{Cov}(X_2,Y)]^2}{\text{Var}(Y)} = 2 - \frac{4}{4} = 1$. This confirms B: $N(1,1)$. Choice A gets the mean right but claims the marginal variance is unchanged — conditioning always reduces (or preserves) variance, never leaves it at the marginal value unless the covariance is zero. Choice C correctly identifies the regression coefficient as $\frac{2}{4} = \frac{1}{2}$, not 1, giving mean $\frac{1}{2}(2)=1$ — but then misreports the mean as 2 and the reasoning is garbled. Choice D confuses this with a deterministic relationship; $X_1 + X_3 = 2$ does not pin down $X_2$ exactly since $X_2$ is not a function of $X_1 + X_3$. Your study tip: always convert "conditional on a linear combination" problems into a two-variable normal problem — compute variance and covariance of the new variable, then apply the standard conditional formulas mechanically.
2. Let $U$ and $V$ be independent $\operatorname{Uniform}(0,1)$ variables, define $W=(U+V)\bmod 1$, and set $X_1=\Phi^{-1}(U)$, $X_2=\Phi^{-1}(V)$, and $X_3=\Phi^{-1}(W)$, where $\Phi$ is the standard normal distribution function. Which conclusion is valid?
   Answer: Every pair is independent bivariate standard normal, but the full vector is not trivariate normal because $X_3$ is a nonlinear function of $(X_1,X_2)$. This question tests a subtle but important distinction in multivariate statistics: pairwise normality does not imply joint normality, and joint normality does not follow from marginal normality alone. Start by examining each component. Since $U, V \sim \text{Uniform}(0,1)$, applying $\Phi^{-1}$ to each yields $X_1, X_2, X_3 \sim N(0,1)$ — that's guaranteed by the probability integral transform. Now, $W = (U+V) \bmod 1$ is itself $\text{Uniform}(0,1)$ (a classical result: the mod-1 sum of independent uniforms is uniform), so $X_3$ is indeed standard normal. For pairwise joint normality: $(X_1, X_2)$ are independent standard normals — trivially bivariate normal. For $(X_1, X_3)$ and $(X_2, X_3)$, one can verify (via characteristic functions or direct argument) that these pairs are also bivariate standard normal, though with nonzero dependence. So A is correct: every pair is bivariate standard normal, yet the full trivariate vector is not jointly normal because $X_3 = \Phi^{-1}((U+V) \bmod 1)$ is a nonlinear, non-affine function of $(X_1, X_2)$ — and a trivariate normal would require all linear combinations to be normal, which fails here. B commits the classic error of assuming pairwise normality upgrades to full joint normality — it does not. C is partially right ($X_3$ is a nonlinear function) but wrong to say $(X_1, X_3)$ is not jointly normal — it actually is bivariate normal, which is the deeper subtlety. D is wrong because there's no deterministic linear relation among the components; the covariance matrix is full rank. Study tip: Pairwise bivariate normality is necessary but not sufficient for joint normality — always a reliable trap on graduate-level exams.
3. Let $X=(X_1,X_2,X_3)^{\mathsf T}$ be zero-mean multivariate normal with covariance matrix $\Sigma=\begin{pmatrix}4&2&0\\2&2&1\\0&1&2\end{pmatrix}.$ Which residual is independent of $(X_2,X_3)^{\mathsf T}$, and what is its variance?
   Answer: $R=X_1-\frac{4}{3}X_2+\frac{2}{3}X_3$ is independent of the predictors and has variance $4/3$. When a question asks which linear combination of predictors leaves a "residual" independent of $(X_2, X_3)^{\mathsf T}$, you should immediately think of the conditional expectation in multivariate normal distributions. For a jointly normal vector, the residual $R = X_1 - \boldsymbol{\beta}^{\mathsf T}(X_2, X_3)^{\mathsf T}$ is independent of $(X_2, X_3)$ if and only if $\boldsymbol{\beta}$ equals the regression coefficient vector from projecting $X_1$ onto $(X_2, X_3)$. To find it, partition $\Sigma$: you need $\boldsymbol{\beta} = \Sigma_{12}\Sigma_{22}^{-1}$, where $\Sigma_{12} = (2, 0)$ and $\Sigma_{22} = \begin{pmatrix}2&1\\1&2\end{pmatrix}$. The inverse is $\Sigma_{22}^{-1} = \frac{1}{3}\begin{pmatrix}2&-1\\-1&2\end{pmatrix}$. Then: $\boldsymbol{\beta} = (2,\,0)\cdot\frac{1}{3}\begin{pmatrix}2&-1\\-1&2\end{pmatrix} = \frac{1}{3}(4,\,-2) = \left(\tfrac{4}{3},\,-\tfrac{2}{3}\right).$ So the residual is $R = X_1 - \tfrac{4}{3}X_2 + \tfrac{2}{3}X_3$, matching answer C. Its variance is the conditional variance: $\text{Var}(R) = \Sigma_{11} - \Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21} = 4 - (2,0)\cdot\tfrac{1}{3}(4,-2)^{\mathsf T} = 4 - \tfrac{8}{3} = \tfrac{4}{3}$. A uses $\boldsymbol{\beta} = (1, 0)$, which doesn't satisfy the projection equations — the residual remains correlated with $(X_2, X_3)$. B gets the signs right conceptually but uses incorrect coefficient values that don't solve $\Sigma_{22}\boldsymbol{\beta} = \Sigma_{21}$. D has the correct magnitude but the wrong sign on $X_3$ — a classic trap since off-diagonal signs in $\Sigma_{22}^{-1}$ flip relative to $\Sigma_{22}$. Always double-check signs when inverting a $2\times2$ matrix — that's exactly where these distractors hide.
4. A parameter vector has prior distribution $\theta\sim N_2(0,\Sigma)$, where $\Sigma=\begin{pmatrix}1&1/2\\1/2&2\end{pmatrix}.$ An observation is generated by $Y=\theta_1+\theta_2+\varepsilon$, where $\varepsilon\sim N(0,1)$ is independent of $\theta$. After observing $Y=5$, which posterior mean and covariance are correct?
   Answer: Mean $(3/2,\,5/2)^{\mathsf T}$ and covariance $\begin{pmatrix}11/20&-1/4\\-1/4&3/4\end{pmatrix}$. Whenever you see a Bayesian updating problem with a multivariate normal prior and a linear observation model, your instinct should be to apply the normal-normal conjugate update formulas directly. The setup: $\theta \sim N_2(0, \Sigma)$ with $\Sigma = \begin{pmatrix}1 & 1/2 \\ 1/2 & 2\end{pmatrix}$, and $Y = c^\top\theta + \varepsilon$ where $c = (1,1)^\top$ and $\varepsilon \sim N(0,1)$. The marginal variance of $Y$ is $c^\top \Sigma c + 1 = (1+1/2+1/2+2)+1 = 5$. The cross-covariance is $\Sigma c = (3/2, 5/2)^\top$. The posterior mean formula gives $\mu_{\text{post}} = \Sigma c \cdot (c^\top \Sigma c + 1)^{-1} \cdot Y = (3/2, 5/2)^\top \cdot (5/5) = (3/2, 5/2)^\top$. The posterior covariance is $\Sigma_{\text{post}} = \Sigma - \Sigma c(c^\top \Sigma c+1)^{-1}c^\top\Sigma = \Sigma - \frac{1}{5}(3/2,5/2)^\top(3/2,5/2)$, which yields $\begin{pmatrix}11/20 & -1/4 \\ -1/4 & 3/4\end{pmatrix}$. This confirms A is correct. Choice B has the right posterior mean but leaves the covariance unchanged as the prior — it ignores that observing data shrinks uncertainty. Choice C uses a wrong scaling: dividing $\Sigma c$ by $c^\top\Sigma c$ alone (forgetting to add the noise variance 1), inflating the update. Choice D applies an incorrect formula entirely, producing a covariance that doesn't match any consistent Bayesian update. Study tip: Always remember the posterior covariance formula subtracts a rank-one matrix from the prior covariance — if your posterior covariance equals the prior, you made no update at all.
5. Let $X_1,\ldots,X_n$ be independent observations from $N_p(\mu,\Sigma)$, where $\Sigma$ is unknown and positive definite. Define $S=(n-1)^{-1}\sum_{i=1}^n(X_i-\bar X)(X_i-\bar X)^{\mathsf T}$ and $T^2=n(\bar X-\mu)^{\mathsf T}S^{-1}(\bar X-\mu)$. Assuming $n>p$, which exact null distribution is correct?
   Answer: $\frac{n-p}{p(n-1)}T^2\sim F_{p,n-p}$. Whenever you encounter Hotelling's $T^2$ statistic in a multivariate normal setting, your goal is to track how it connects to a known pivot distribution through the relationship between the Wishart and F distributions. Start from first principles: $\sqrt{n}(\bar{X}-\mu)\sim N_p(0,\Sigma)$, and $(n-1)S\sim W_p(\Sigma, n-1)$ independently. Hotelling showed that $T^2 = n(\bar{X}-\mu)^\mathsf{T}S^{-1}(\bar{X}-\mu)$ follows a $T^2_{p,n-1}$ distribution. The key bridge to the F distribution is the exact result: $\frac{n-p}{p(n-1)}T^2 \sim F_{p,\,n-p}$ This is answer A, and it is correct. The degrees of freedom $p$ in the numerator reflect the dimension of the mean vector, while $n-p$ in the denominator comes from $n-1$ (Wishart df) minus $(p-1)$, a consequence of the multivariate pivot construction. Answer B is wrong on two counts: the scaling uses $pn$ instead of $p(n-1)$, and the denominator degrees of freedom $n-p+1$ has no basis in the correct derivation. Answer C is a common and tempting trap. While $n(\bar{X}-\mu)^\mathsf{T}\Sigma^{-1}(\bar{X}-\mu)\sim\chi^2_p$ when $\Sigma$ is known, replacing $\Sigma$ with the estimated $S$ introduces additional variability that prevents a chi-squared result — you need the F distribution to account for this estimation error. Answer D flips the degrees of freedom, writing $F_{n-p,\,p}$ instead of $F_{p,\,n-p}$, which reverses numerator and denominator and gives an entirely different distribution. Study tip: Memorize the exact scaling constant $\frac{n-p}{p(n-1)}$ for Hotelling's $T^2\to F_{p,n-p}$, and always remember that substituting an estimated covariance matrix eliminates the chi-squared pivot — that distinction is frequently tested.
6. Let $X\sim N_4(\mu,I_4)$ with $\mu=(1,1,0,0)^{\mathsf T}$, and define $Q=\frac{(X_1+X_2)^2}{2}+\frac{(X_3-X_4)^2}{2}.$ What is the distribution of $Q$?
   Answer: A noncentral chi-square distribution with $2$ degrees of freedom and noncentrality parameter $2$. When you encounter a quadratic form built from normal random variables, your first move should be to rewrite it as a sum of squared independent standard normals — that's the blueprint for identifying noncentral chi-square distributions. Define two new variables: $Y_1 = \frac{X_1 + X_2}{\sqrt{2}}$ and $Y_2 = \frac{X_3 - X_4}{\sqrt{2}}$. Since $X \sim N_4(\mu, I_4)$, these are independent normal random variables (the linear combinations involve disjoint, orthogonal coordinate pairs). Their means are $E[Y_1] = \frac{1+1}{\sqrt{2}} = \sqrt{2}$ and $E[Y_2] = \frac{0-0}{\sqrt{2}} = 0$, and both have variance 1. So $Q = Y_1^2 + Y_2^2$, the sum of squares of two independent normals with unit variance. This is exactly a noncentral chi-square with 2 degrees of freedom. The noncentrality parameter is $\lambda = \mu_{Y_1}^2 + \mu_{Y_2}^2 = (\sqrt{2})^2 + 0^2 = 2$. The correct answer is B. A is wrong because it claims 4 degrees of freedom — a tempting trap since $X$ is 4-dimensional, but $Q$ is built from only 2 independent squared terms. C gets the degrees of freedom right but miscalculates $\lambda$; doubling it to 4 would require $\mu = (1,1,1,1)^T$ or similar. D is wrong on two counts: orthogonality of the contrasts doesn't erase noncentrality, and $Q$ is noncentral because $Y_1$ has nonzero mean. Study tip: Always compute the noncentrality parameter as $\lambda = \sum_i \mu_{Z_i}^2$ after standardizing — don't confuse the original $\mu$ components with $\lambda$ directly.
7. Let $(X_1,X_2)^{\mathsf T}$ have a nonsingular bivariate normal distribution with arbitrary means, variances $\sigma_1^2$ and $\sigma_2^2$, and covariance $\tau$. Define $U=X_1+X_2$ and $V=X_1-X_2$. Which condition is necessary and sufficient for $U$ and $V$ to be independent?
   Answer: The original variables must have equal variances, so the necessary and sufficient condition is $\sigma_1^2=\sigma_2^2$. When dealing with linear combinations of jointly normal random variables, the key tool is the covariance criterion: two jointly normal random variables are independent if and only if they are uncorrelated. So your strategy is to compute $\text{Cov}(U, V)$ and set it to zero. Since $U = X_1 + X_2$ and $V = X_1 - X_2$, expand the covariance directly: $\text{Cov}(U, V) = \text{Cov}(X_1 + X_2,\, X_1 - X_2) = \text{Var}(X_1) - \text{Var}(X_2) = \sigma_1^2 - \sigma_2^2$ (The cross-terms involving $\tau$ cancel: $+\tau - \tau = 0$.) Setting this to zero gives $\sigma_1^2 = \sigma_2^2$, confirming that C is the necessary and sufficient condition — equal variances. A is wrong because $\tau$ drops out entirely from $\text{Cov}(U,V)$; whether or not $X_1$ and $X_2$ are uncorrelated has no bearing on the independence of $U$ and $V$. This is a tempting distractor because uncorrelatedness is often a key condition, but here the algebra shows it's irrelevant. B is wrong because means have no effect on covariance whatsoever — equal means is neither necessary nor sufficient for anything related to independence in this context. D is wrong because it sets $\text{Cov}(U,V) = \sigma_1^2 - \sigma_2^2 + \text{something involving}\,\tau$ to zero incorrectly; the actual covariance contains no $\tau$ term, so introducing $\tau$ into the condition is a miscalculation. Study tip: When checking independence of linear combinations of normals, always compute the covariance explicitly — don't guess which original parameters matter. Let the algebra decide.
8. Let $(X,Y)^{\mathsf T}$ be standard bivariate normal with correlation $1/2$. The distribution is truncated by retaining only observations for which $Y>0$. Which statement about the conditional distribution of $X\mid Y>0$ is correct?
   Answer: It is generally nonnormal, with mean $\frac{1}{2}\sqrt{\frac{2}{\pi}}$ and variance $1-\frac{1}{2\pi}$. When a bivariate normal vector is truncated (not conditioned on an exact value), the marginal distribution of the non-truncated variable becomes a mixture over a range, generally producing a non-normal shape. That's the key conceptual wedge here. Start with the joint structure. Since $(X,Y)^T$ is standard bivariate normal with correlation $\rho = 1/2$, you can write $X = \frac{1}{2}Y + \sqrt{\frac{3}{4}}\,Z$ where $Z \perp Y$. Now condition on $Y > 0$. The mean becomes $E[X \mid Y > 0] = \frac{1}{2}E[Y \mid Y > 0] = \frac{1}{2}\cdot\sqrt{\frac{2}{\pi}}$, since the truncated standard normal has mean $\sqrt{2/\pi}$. For variance, use the law of total variance: $\text{Var}(X \mid Y>0) = \frac{1}{4}\text{Var}(Y\mid Y>0) + \frac{3}{4}$. The truncated standard normal has variance $1 - \frac{2}{\pi}$, so the total is $\frac{1}{4}\!\left(1-\frac{2}{\pi}\right)+\frac{3}{4} = 1 - \frac{1}{2\pi}$. This confirms B is correct — and the distribution is non-normal because it's a mixture of normals with different means (indexed by truncated $Y$). A is wrong on variance: truncation does affect variance through the correlation, reducing it below 1, not locking it at $3/4$. C gets the mean right but uses $3/4$ for the variance, forgetting to add $\frac{1}{4}\text{Var}(Y\mid Y>0)$— it's the conditional variance of $Z$'s contribution only, not the full answer. D confuses truncation with independence; correlating $X$ with $Y$ means restricting $Y$'s range genuinely shifts the distribution of $X$. Your study tip: always decompose the truncated variable using the bivariate normal regression form, then apply the law of total variance carefully — the truncated component's variance is strictly less than 1, and that reduction propagates.
9. A zero-mean trivariate normal vector has precision matrix $\Omega=\Sigma^{-1}=\begin{pmatrix}2&-1&0\\-1&2&-1\\0&-1&2\end{pmatrix}.$ Which statement correctly describes the relationship between $X_1$ and $X_3$?
   Answer: They are conditionally independent given $X_2$ and marginally positively correlated, with correlation $1/3$. Whenever you see a precision matrix (the inverse of the covariance matrix) in a multivariate normal problem, you need to keep two distinct facts sharp in your mind: **zero off-diagonal entries in $\Omega$ signal conditional independence, while marginal independence requires zero entries in $\Sigma$ itself** — and these are not the same thing. Here, $\Omega_{13} = 0$ tells you immediately that $X_1 \perp X_3 \mid X_2$ — they are conditionally independent given $X_2$. That confirms the first half of answer D. For the marginal covariance, you must actually invert $\Omega$. Computing $\Sigma = \Omega^{-1}$, the $(1,3)$ entry works out to $\Sigma_{13} = 1/4$... wait — let's be precise. For this tridiagonal matrix, the inverse gives $\Sigma_{13} = \Sigma_{31} = \frac{1}{4}$, which is strictly positive. The marginal correlation is $\frac{\Sigma_{13}}{\sqrt{\Sigma_{11}\Sigma_{33}}}$, and since all diagonal entries equal $3/4$ by symmetry, this gives correlation $\frac{1/4}{3/4} = 1/3 > 0$. So $X_1$ and $X_3$ are marginally positively correlated, confirming D is correct. A gets the conditional/marginal relationship backwards — they are conditionally independent (not correlated) and marginally correlated (not independent). B is the most dangerous trap: it conflates $\Omega_{13} = 0$ with both conditional and marginal independence, ignoring that the inverse matrix generally has no zeros where the original does. C makes the same error as B in the second clause — $\Omega_{13} = 0$ says nothing directly about $\Sigma_{13}$. Study tip: Tattoo this onto your brain — zeros in $\Omega$ mean conditional independence; zeros in $\Sigma$ mean marginal independence. You must invert to check the other.
10. Suppose $X=(X_1,X_2,X_3)^{\mathsf T}$ is multivariate normal with mean $\mu=(1,-1,0)^{\mathsf T}$ and covariance matrix $\Sigma=\begin{pmatrix}2&1&3\\1&2&3\\3&3&6\end{pmatrix}.$ Which statement is correct?
    Answer: $P(X_3=X_1+X_2)=1$, and the joint distribution has no density with respect to three-dimensional Lebesgue measure. When a multivariate normal vector has a singular covariance matrix, its distribution collapses onto a lower-dimensional affine subspace — this is the core concept being tested here. Start by checking whether $\Sigma$ is singular. Notice that column 3 of $\Sigma$ equals column 1 plus column 2: $(3,3,6)^T = (2,1,3)^T + (1,2,3)^T$. This means $\det(\Sigma) = 0$, so $\Sigma$ is singular. Now find what linear constraint this imposes. Since $\text{Cov}(X_3 - X_1 - X_2) = \text{Var}(X_3) + \text{Var}(X_1) + \text{Var}(X_2) - 2\text{Cov}(X_1,X_3) - 2\text{Cov}(X_2,X_3) + 2\text{Cov}(X_1,X_2) = 6 + 2 + 2 - 6 - 6 + 2 = 0$, the random variable $X_3 - X_1 - X_2$ is degenerate (zero variance). Its mean is $E[X_3 - X_1 - X_2] = 0 - 1 - (-1) = 0$. A normal random variable with mean 0 and variance 0 equals 0 almost surely, so $P(X_3 = X_1 + X_2) = 1$. Because the distribution is confined to a proper affine subspace of $\mathbb{R}^3$, it has no density with respect to 3D Lebesgue measure. This confirms C is correct. A is wrong because it misidentifies the constraint as $X_3 = X_1 - X_2$; computing $E[X_1 - X_2 - X_3] = 1-(-1)-0 = 2 \neq 0$, so that relation fails. B is wrong because continuous marginals don't prevent joint degeneracy — each $X_i$ individually can be continuous while the joint distribution is singular. D is a fabricated distractor; probabilities of exact linear constraints for jointly normal variables are either 0 or 1, never 1/2. Study tip: Whenever you see a multivariate normal question, immediately check if $\Sigma$ is singular by looking for linearly dependent rows/columns. If it is, identify the null-space direction — that tells you the exact almost-sure linear constraint the vector satisfies.

Start this quiz