Statistics Graduate Level Quiz: Covariance Matrices And Eigen Decomposition
10 questions · exam conditions
0:00
Covariance Matrices And Eigen DecompositionQuestion 1 of 10

Four standardized measurements have the exchangeable correlation matrix R=(1−ρ)I4+ρ11TR=(1-\rho)I_4+\rho\mathbf{1}\mathbf{1}^{\mathsf T}, where ρ=−0.2\rho=-0.2.

Which statement correctly describes the leading principal-component eigenspace?

It is the span of 1\mathbf{1}, and its eigenvalue is 1.21.2.
It is the span of 1\mathbf{1}, and its eigenvalue is 0.40.4.
It is the three-dimensional contrast space orthogonal to 1\mathbf{1}, with eigenvalue 1.21.2.
It is the three-dimensional contrast space orthogonal to 1\mathbf{1}, with eigenvalue 0.40.4.
← Back to quizzes

Statistics Graduate Level Quiz

Statistics Graduate Level Quiz: Covariance Matrices And Eigen Decomposition

Practice Covariance Matrices And Eigen Decomposition in Statistics Graduate Level with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Covariance Matrices And Eigen Decomposition, giving you a quick way to practice the rules, question types, and explanations that matter most for Statistics Graduate Level.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

Four standardized measurements have the exchangeable correlation matrix R=(1−ρ)I4+ρ11TR=(1-\rho)I_4+\rho\mathbf{1}\mathbf{1}^{\mathsf T}, where ρ=−0.2\rho=-0.2.

Which statement correctly describes the leading principal-component eigenspace?

  1. It is the span of 1\mathbf{1}, and its eigenvalue is 1.21.2.
  2. It is the span of 1\mathbf{1}, and its eigenvalue is 0.40.4.
  3. It is the three-dimensional contrast space orthogonal to 1\mathbf{1}, with eigenvalue 1.21.2. (correct answer)
  4. It is the three-dimensional contrast space orthogonal to 1\mathbf{1}, with eigenvalue 0.40.4.
Explanation: When you encounter a question about PCA on an exchangeable correlation matrix, your first move should be to find the eigenvalues analytically using the known structure of R=(1−ρ)I4+ρ11TR = (1-\rho)I_4 + \rho\mathbf{1}\mathbf{1}^\mathsf{T}. This matrix has exactly two distinct eigenvalues. The vector 1\mathbf{1} is an eigenvector with eigenvalue λ1=(1−ρ)+4ρ=1+3ρ\lambda_1 = (1-\rho) + 4\rho = 1 + 3\rho. Any vector orthogonal to 1\mathbf{1} (a "contrast vector") is an eigenvector with eigenvalue λ2=1−ρ\lambda_2 = 1 - \rho. Plugging in ρ=−0.2\rho = -0.2: λ1=1+3(−0.2)=0.4\lambda_1 = 1 + 3(-0.2) = 0.4 λ2=1−(−0.2)=1.2\lambda_2 = 1 - (-0.2) = 1.2 Since 1.2>0.41.2 > 0.4, the leading principal component corresponds to λ2=1.2\lambda_2 = 1.2, whose eigenspace is the three-dimensional space of all vectors orthogonal to 1\mathbf{1} — the contrast space. That confirms C is correct. Answer A gets the eigenspace wrong — it claims 1\mathbf{1} spans the leading component — and assigns it 1.21.2, which is actually the contrast eigenvalue. Answer B correctly identifies that 1\mathbf{1} has eigenvalue 0.40.4, but calling it the leading component is the critical error; it's actually the smallest eigenvalue here. Answer D correctly identifies the contrast space but assigns it the wrong eigenvalue, 0.40.4 instead of 1.21.2. The key study tip: with negative ρ\rho, the roles of 1\mathbf{1} and the contrast space flip compared to the positive-ρ\rho case. Always compute both eigenvalues before deciding which dominates.

Question 2

A mean-zero random vector has positive-definite covariance matrix Σ=QΛQT\Sigma=Q\Lambda Q^{\mathsf T}, where the columns of QQ are ordered orthonormal eigenvectors and Λ\Lambda contains the corresponding eigenvalues.

For Z=WXZ=WX, which transformation both gives Cov⁡(Z)=I\operatorname{Cov}(Z)=I and makes the kkth component of ZZ a rescaled projection onto the kkth eigenvector?

  1. W=Λ−1/2QTW=\Lambda^{-1/2}Q^{\mathsf T} (correct answer)
  2. W=QΛ−1/2QTW=Q\Lambda^{-1/2}Q^{\mathsf T}
  3. W=QTΛ−1/2W=Q^{\mathsf T}\Lambda^{-1/2}
  4. W=Λ1/2QTW=\Lambda^{1/2}Q^{\mathsf T}
Explanation: When whitening a random vector, you need a transformation WW such that Cov(WX)=WΣWT=I\text{Cov}(WX) = W\Sigma W^{\mathsf T} = I. The spectral decomposition Σ=QΛQT\Sigma = Q\Lambda Q^{\mathsf T} gives you everything you need — but the order of operations matters critically. Choice A, W=Λ−1/2QTW = \Lambda^{-1/2}Q^{\mathsf T}, is correct. Verify it: WΣWT=Λ−1/2QT(QΛQT)QΛ−1/2=Λ−1/2(QTQ)Λ(QTQ)Λ−1/2=Λ−1/2ΛΛ−1/2=IW\Sigma W^{\mathsf T} = \Lambda^{-1/2}Q^{\mathsf T}(Q\Lambda Q^{\mathsf T})Q\Lambda^{-1/2} = \Lambda^{-1/2}(Q^{\mathsf T}Q)\Lambda(Q^{\mathsf T}Q)\Lambda^{-1/2} = \Lambda^{-1/2}\Lambda\Lambda^{-1/2} = I. Crucially, the kkth component of Z=WXZ = WX is λk−1/2(qkTx)\lambda_k^{-1/2}(q_k^{\mathsf T}x) — a rescaled projection of XX onto the kkth eigenvector qkq_k. This is precisely the PCA score, normalized to unit variance. Choice B, W=QΛ−1/2QTW = Q\Lambda^{-1/2}Q^{\mathsf T}, is the symmetric whitening (Mahalanobis) transform. While Cov(WX)=I\text{Cov}(WX) = I holds here too, the components of ZZ are not projections onto individual eigenvectors — they're linear combinations of all eigenvectors, failing the second requirement. Choice C, W=QTΛ−1/2W = Q^{\mathsf T}\Lambda^{-1/2}, reverses the order of QTQ^{\mathsf T} and Λ−1/2\Lambda^{-1/2}. Computing WΣWTW\Sigma W^{\mathsf T} yields QTΛ−1/2QΛQTΛ−1/2QQ^{\mathsf T}\Lambda^{-1/2}Q\Lambda Q^{\mathsf T}\Lambda^{-1/2}Q, which does not simplify to II because Λ−1/2\Lambda^{-1/2} and QQ don't commute. Choice D, W=Λ1/2QTW = \Lambda^{1/2}Q^{\mathsf T}, uses Λ1/2\Lambda^{1/2} instead of Λ−1/2\Lambda^{-1/2}, which would amplify variance rather than normalize it, giving Cov(Z)=Λ2≠I\text{Cov}(Z) = \Lambda^2 \neq I. Your strategy: always verify whitening candidates by computing WΣWTW\Sigma W^{\mathsf T} explicitly, and separately check whether components correspond to individual eigenvector projections — both conditions must hold simultaneously.

Question 3

Two variables have covariance matrix $$\Sigma=\begin{pmatrix}9&3\3&4\end{pmatrix}

Which expression gives the first standardized principal-component score in terms of the original variables, together with its variance?

  1. The score is X132+X222\frac{X_1}{3\sqrt{2}}+\frac{X_2}{2\sqrt{2}}, and its variance is 32\frac{3}{2}. (correct answer)
  2. The score is X1+X22\frac{X_1+X_2}{\sqrt{2}}, and its variance is 32\frac{3}{2}.
  3. The score is X132−X222\frac{X_1}{3\sqrt{2}}-\frac{X_2}{2\sqrt{2}}, and its variance is 12\frac{1}{2}.
  4. The score is X122+X232\frac{X_1}{2\sqrt{2}}+\frac{X_2}{3\sqrt{2}}, and its variance is 32\frac{3}{2}.
Explanation: When PCA is performed on standardized variables, you're working with the correlation matrix, not the original covariance matrix. The standardized variables are Z1=X1/3Z_1 = X_1/3 and Z2=X2/2Z_2 = X_2/2 (dividing by the standard deviations 9=3\sqrt{9}=3 and 4=2\sqrt{4}=2). The correlation between them is ρ=3/(3⋅2)=1/2\rho = 3/(3 \cdot 2) = 1/2, giving correlation matrix $$R = \begin{pmatrix}1 & 1/2 \ 1/2 & 1\end{pmatrix} The eigenvalues of $$R$$ are $$\lambda_1 = 3/2$$ and $$\lambda_2 = 1/2$$. The first eigenvector (for $$\lambda_1 = 3/2$$) is $$\mathbf{e}_1 = (1/\sqrt{2}, 1/\sqrt{2})^\top$$. The first **standardized** principal component score is therefore $$PC_1 = \frac{Z_1}{\sqrt{2}} + \frac{Z_2}{\sqrt{2}} = \frac{X_1}{3\sqrt{2}} + \frac{X_2}{2\sqrt{2}}$$, with variance equal to the eigenvalue $$\lambda_1 = 3/2$$. This confirms **A** is correct. **B** is wrong because it uses $$X_1$$ and $$X_2$$ directly without standardizing — it treats the original variables as if they already had unit variance. **C** applies the correct coefficients but uses a minus sign, corresponding to the *second* eigenvector (for $$\lambda_2 = 1/2$$), not the first. **D** swaps the denominators — it divides $$X_1$$ by $$2\sqrt{2}$$ and $$X_2$$ by $$3\sqrt{2}$$, reversing which standard deviation belongs to which variable. **Study tip:** Always remember that standardized PCA operates on the correlation matrix $$R$$, not $$\Sigma$$. Translate eigenvector scores back to original variables by dividing each $$X_i$$ by its own standard deviation $$\sigma_i$$ — mixing up those denominators is the most common trap here.

Question 4

In a centered linear regression with sample size nn, the predictor cross-product matrix is XTX=nΣX^{\mathsf T}X=n\Sigma, where the smallest eigenvalue of Σ\Sigma is 1/41/4. Errors have covariance σ2I\sigma^2I. Ridge regression uses penalty λ=n/4\lambda=n/4 in the estimator (XTX+λI)−1XTy(X^{\mathsf T}X+\lambda I)^{-1}X^{\mathsf T}y.

Along the eigenvector associated with the smallest eigenvalue of Σ\Sigma, what is the ratio of the sampling variance of the ridge coefficient to that of the ordinary least-squares coefficient?

  1. The variance ratio is 1/81/8.
  2. The variance ratio is 1/21/2.
  3. The variance ratio is 1/41/4. (correct answer)
  4. The variance ratio is 1/161/16.
Explanation: When comparing ridge and OLS variance along a specific eigendirection, the key is to work in the eigenspace of XTXX^\mathsf{T}X and track how each estimator scales the signal. For OLS, the covariance matrix is σ2(XTX)−1\sigma^2(X^\mathsf{T}X)^{-1}. Along an eigenvector with eigenvalue ϕ\phi of XTXX^\mathsf{T}X, the OLS variance is σ2/ϕ\sigma^2/\phi. For ridge, the covariance is σ2(XTX+λI)−1(XTX)(XTX+λI)−1\sigma^2(X^\mathsf{T}X+\lambda I)^{-1}(X^\mathsf{T}X)(X^\mathsf{T}X+\lambda I)^{-1}, giving variance σ2ϕ/(ϕ+λ)2\sigma^2\phi/(\phi+\lambda)^2 along that same eigenvector. The variance ratio (ridge to OLS) is therefore: σ2ϕ/(ϕ+λ)2σ2/ϕ=ϕ2(ϕ+λ)2\frac{\sigma^2\phi/(\phi+\lambda)^2}{\sigma^2/\phi} = \frac{\phi^2}{(\phi+\lambda)^2} Now plug in the numbers. The smallest eigenvalue of Σ\Sigma is 1/41/4, and XTX=nΣX^\mathsf{T}X = n\Sigma, so the corresponding eigenvalue of XTXX^\mathsf{T}X is ϕ=n/4\phi = n/4. With λ=n/4\lambda = n/4: ϕ2(ϕ+λ)2=(n/4)2(n/4+n/4)2=(n/4)2(n/2)2=14\frac{\phi^2}{(\phi+\lambda)^2} = \frac{(n/4)^2}{(n/4+n/4)^2} = \frac{(n/4)^2}{(n/2)^2} = \frac{1}{4} This confirms C is correct. Choice A (1/81/8) has no algebraic basis here — it might tempt you if you mistakenly take a cube rather than square. Choice B (1/21/2) is the ratio ϕ/(ϕ+λ)\phi/(\phi+\lambda), which is only one factor of the correct squared expression — a common half-finished calculation error. Choice D (1/161/16) results from squaring the wrong base ratio, perhaps computing (1/4)2(1/4)^2 directly without correctly forming ϕ/(ϕ+λ)\phi/(\phi+\lambda). Study tip: Always derive the ridge variance formula from scratch using the sandwich form — memorizing only the OLS formula and forgetting the extra ϕ\phi in the numerator is the most frequent trap on problems like this.

Question 5

Three standardized random variables have pairwise correlations Corr⁡(X,Y)=Corr⁡(Y,Z)=ρ\operatorname{Corr}(X,Y)=\operatorname{Corr}(Y,Z)=\rho and Corr⁡(X,Z)=0\operatorname{Corr}(X,Z)=0, where ρ≥0\rho\ge 0. What is the largest value of ρ\rho for which these correlations can define a valid correlation matrix?

  1. ρ=12\rho=\frac{1}{2}
  2. ρ=12\rho=\frac{1}{\sqrt{2}} (correct answer)
  3. ρ=32\rho=\frac{\sqrt{3}}{2}
  4. ρ=1\rho=1
Explanation: When you encounter a question about valid correlation structures, your first instinct should be to think about positive semi-definiteness. A correlation matrix is valid if and only if it is positive semi-definite (PSD), meaning all its eigenvalues must be non-negative. The correlation matrix for (X,Y,Z)(X, Y, Z) with the given structure is: For RR to be PSD, its determinant must be non-negative (a necessary condition that is also sufficient here alongside checking leading minors). Computing: det⁡(R)=1(1−ρ2)−ρ(ρ−0)+0=1−ρ2−ρ2=1−2ρ2\det(R) = 1(1 - \rho^2) - \rho(\rho - 0) + 0 = 1 - \rho^2 - \rho^2 = 1 - 2\rho^2 Setting det⁡(R)≥0\det(R) \geq 0 gives ρ2≤12\rho^2 \leq \frac{1}{2}, so ρ≤12\rho \leq \frac{1}{\sqrt{2}}. This confirms answer B is correct — the largest valid ρ\rho is exactly 12\frac{1}{\sqrt{2}}. As for the distractors: A (ρ=12\rho = \frac{1}{2}) is a valid correlation but is not the maximum — larger values up to 12\frac{1}{\sqrt{2}} are also achievable, so it fails the "largest" criterion. C (ρ=32\rho = \frac{\sqrt{3}}{2}) yields det⁡(R)=1−2(3/4)=−12<0\det(R) = 1 - 2(3/4) = -\frac{1}{2} < 0, making the matrix invalid. D (ρ=1\rho = 1) would force X=Y=ZX = Y = Z almost surely, directly contradicting Corr⁡(X,Z)=0\operatorname{Corr}(X,Z) = 0. Study tip: Whenever a question asks for the feasible range of a correlation parameter, immediately write out the correlation matrix and apply the PSD conditions — start with the determinant, then check all leading principal minors.

Question 6

In a two-class linear discriminant analysis, the within-class covariance matrix is W=diag⁡(4,1)W=\operatorname{diag}(4,1). The between-class covariance is proportional to vvTvv^{\mathsf T}, where v=(2,1)Tv=(2,1)^{\mathsf T}; take the constant of proportionality to be 11.

For the generalized eigenproblem Ba=λWaB a=\lambda W a, which pair gives the nonzero generalized eigenvalue and a corresponding direction?

  1. The eigenvalue is 22, with direction proportional to (1/2,1)T(1/2,1)^{\mathsf T}. (correct answer)
  2. The eigenvalue is 55, with direction proportional to (2,1)T(2,1)^{\mathsf T}.
  3. The eigenvalue is 11, with direction proportional to (1,1/2)T(1,1/2)^{\mathsf T}.
  4. The eigenvalue is 5/45/4, with direction proportional to (1/2,1/4)T(1/2,1/4)^{\mathsf T}.
Explanation: When you encounter a generalized eigenproblem Ba=λWaBa = \lambda Wa in LDA, the key insight is that you can convert it to a standard eigenproblem by computing W−1BW^{-1}B, then find eigenvalues and eigenvectors normally. Here, B=vvT=(4221)B = vv^\mathsf{T} = \begin{pmatrix}4&2\\2&1\end{pmatrix} and W−1=diag(1/4,1)W^{-1} = \text{diag}(1/4, 1), so: Since BB is rank-1, there is exactly one nonzero eigenvalue. The trace gives λ=1+1=2\lambda = 1 + 1 = 2. For the eigenvector, solve (W−1B−2I)a=0(W^{-1}B - 2I)a = 0: the system gives −a1+12a2=0-a_1 + \frac{1}{2}a_2 = 0, so a1=12a2a_1 = \frac{1}{2}a_2, yielding direction (1/2,1)T(1/2, 1)^\mathsf{T}. This confirms answer A is correct. Answer B is tempting because (2,1)T(2,1)^\mathsf{T} is the mean-difference vector vv itself, but the LDA direction is W−1vW^{-1}v, not vv. Plugging into the eigenproblem, W−1B(2,1)T=(2,5)TW^{-1}B(2,1)^\mathsf{T} = (2,5)^\mathsf{T}, which is not a scalar multiple of (2,1)T(2,1)^\mathsf{T}, so it fails. Answer C uses direction (1,1/2)T(1,1/2)^\mathsf{T} with eigenvalue 1 — neither checks out against W−1BW^{-1}B. Answer D offers a scaled version of vv with a fractional eigenvalue, which is simply numerically inconsistent with the trace calculation. A useful habit: always verify a candidate eigenpair by checking W−1Ba=λaW^{-1}Ba = \lambda a directly — it takes seconds and eliminates traps built around plausible-looking but incorrect directions.

Question 7

The jointly Gaussian vector (X1,X2,Y)T(X_1,X_2,Y)^{\mathsf T} has Cov⁡(X1,X2)=(2112)\operatorname{Cov}(X_1,X_2)=\begin{pmatrix}2&1\\1&2\end{pmatrix} , Cov⁡((X1,X2)T,Y)=(1,1)T\operatorname{Cov}((X_1,X_2)^{\mathsf T},Y)=(1,1)^{\mathsf T}, and Var⁡(Y)=2\operatorname{Var}(Y)=2.

What are the eigenvalues of the conditional covariance matrix of (X1,X2)T(X_1,X_2)^{\mathsf T} given YY?

  1. The conditional eigenvalues are 33 and 11.
  2. The conditional eigenvalues are 32\frac{3}{2} and 12\frac{1}{2}.
  3. The conditional eigenvalues are 22 and 00.
  4. The conditional eigenvalues are 22 and 11. (correct answer)
Explanation: When you see a question about conditional distributions in a jointly Gaussian setting, your first instinct should be the standard Gaussian conditioning formula. For a jointly Gaussian vector, the conditional covariance of (X1,X2)T(X_1, X_2)^\mathsf{T} given YY is: ΣX∣Y=ΣXX−ΣXYΣYY−1ΣYX\Sigma_{X|Y} = \Sigma_{XX} - \Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX} Here, ΣXX=(2112)\Sigma_{XX} = \begin{pmatrix}2&1\\1&2\end{pmatrix} , ΣXY=(11)\Sigma_{XY} = \begin{pmatrix}1\\1\end{pmatrix} , and ΣYY=2\Sigma_{YY} = 2. Plugging in: The eigenvalues of (3/21/21/23/2)\begin{pmatrix}3/2&1/2\\1/2&3/2\end{pmatrix} are 3/2+1/2=23/2 + 1/2 = 2 and 3/2−1/2=13/2 - 1/2 = 1, confirming answer D. Choice A (eigenvalues 3 and 1) corresponds to the eigenvalues of the unconditional ΣXX\Sigma_{XX}, forgetting to subtract the correction term entirely. Choice B (eigenvalues 3/2 and 1/2) is a subtle trap: these are the diagonal entries of ΣX∣Y\Sigma_{X|Y}, which students sometimes confuse for eigenvalues — a common mistake when the matrix isn't diagonal. Choice C (eigenvalues 2 and 0) likely arises from incorrectly computing the correction term, perhaps treating ΣXYΣYY−1ΣYX\Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX} as the full original matrix. As a study tip: always distinguish between diagonal entries and eigenvalues. For a symmetric 2×22\times2 matrix (abba)\begin{pmatrix}a&b\\b&a\end{pmatrix}, eigenvalues are a+ba+b and a−ba-b — memorize this pattern to avoid the trap in choice B.

Question 8

Two unknown 2×22\times2 covariance matrices, Σ1\Sigma_1 and Σ2\Sigma_2, each have eigenvalues 44 and 11. Their eigenvectors need not be aligned.

As the relative orientation of their eigenvectors varies, what is the full possible range of the largest eigenvalue of Σ1+Σ2\Sigma_1+\Sigma_2?

  1. The range is from 44 through 88.
  2. The largest eigenvalue is always 88.
  3. The largest eigenvalue is always 55.
  4. The range is from 55 through 88. (correct answer)
Explanation: When two symmetric matrices share the same eigenvalues but have different eigenvectors, their sum's eigenvalues depend critically on how their principal axes are aligned. This is a question about the Weyl inequalities and the geometry of eigenvalue addition. Start by noting that each Σi\Sigma_i has trace 4+1=54 + 1 = 5, so Σ1+Σ2\Sigma_1 + \Sigma_2 always has trace 1010. This means the two eigenvalues of the sum always add to 1010, so finding the largest eigenvalue tells you both. When eigenvectors are aligned, both matrices are simultaneously diagonalizable. The largest eigenvalue of the sum is simply 4+4=84 + 4 = 8, and the smallest is 1+1=21 + 1 = 2. This is the maximum possible for the largest eigenvalue. When eigenvectors are orthogonal (rotated 90°90° relative to each other), one matrix "spreads" variance in the direction the other "compresses" it. You can verify explicitly: if Σ1=diag(4,1)\Sigma_1 = \text{diag}(4,1) and Σ2\Sigma_2 has its large eigenvector along the second axis, then Σ1+Σ2=diag(5,5)\Sigma_1 + \Sigma_2 = \text{diag}(5,5), giving largest eigenvalue 55. This is the minimum. By continuity, as orientation varies continuously between these extremes, the largest eigenvalue achieves every value in [5,8][5, 8], making D correct. A is wrong because it sets the lower bound at 44, which is impossible — the sum of two positive definite matrices must have eigenvalues larger than either individual matrix's minimum. B incorrectly claims the eigenvalues are always maximally aligned. C correctly identifies the minimum but wrongly claims it's the only possibility. Remember: for eigenvalue problems involving sums of matrices, trace constraints give you sum of eigenvalues for free, and extreme orientations (parallel vs. orthogonal eigenvectors) give you the range endpoints.

Question 9

A random vector has covariance matrix with orthonormal eigenvectors q1,q2,q3q_1,q_2,q_3 and corresponding eigenvalues 6,3,16,3,1. Independent calibration error is then added only in the direction u=(q1+q2)/2u=(q_1+q_2)/\sqrt{2}, with error variance 44.

What are the largest eigenvalue and a corresponding unit eigenvector of the covariance matrix after adding the calibration error?

  1. The eigenvalue is 1010, with eigenvector u=(q1+q2)/2u=(q_1+q_2)/\sqrt{2}.
  2. The eigenvalue is 99, with eigenvector (2q1+q2)/5(2q_1+q_2)/\sqrt{5}. (correct answer)
  3. The eigenvalue is 88, with eigenvector q1q_1.
  4. The eigenvalue is 77, with eigenvector (q1+2q2)/5(q_1+2q_2)/\sqrt{5}.
Explanation: When a rank-1 perturbation is added to a covariance matrix, you need to find the new eigenstructure. The updated covariance matrix is Σ′=Σ+4uuT\Sigma' = \Sigma + 4uu^T, where u=(q1+q2)/2u = (q_1+q_2)/\sqrt{2}. Since uuT=12(q1q1T+q1q2T+q2q1T+q2q2T)uu^T = \frac{1}{2}(q_1q_1^T + q_1q_2^T + q_2q_1^T + q_2q_2^T), the perturbation mixes only the q1q_1 and q2q_2 subspace — the q3q_3 direction is unaffected, retaining eigenvalue 1. To find the new eigenvalues in the {q1,q2}\{q_1, q_2\} subspace, express Σ′\Sigma' restricted to that subspace. The original part contributes the diagonal matrix diag(6,3)\text{diag}(6,3), and 4uuT4uu^T contributes 4⋅12(1111)=(2222)4 \cdot \frac{1}{2}\begin{pmatrix}1&1\\1&1\end{pmatrix} = \begin{pmatrix}2&2\\2&2\end{pmatrix} . So the restricted matrix is (8225)\begin{pmatrix}8&2\\2&5\end{pmatrix}. Its eigenvalues satisfy (8−λ)(5−λ)−4=0(8-\lambda)(5-\lambda)-4=0, giving λ2−13λ+36=0\lambda^2 - 13\lambda + 36 = 0, so λ=9\lambda = 9 or λ=4\lambda = 4. For λ=9\lambda=9: (8−9)v1+2v2=0⇒v2=v1/2(8-9)v_1 + 2v_2 = 0 \Rightarrow v_2 = v_1/2, normalized to (2q1+q2)/5(2q_1+q_2)/\sqrt{5}. This confirms answer B — eigenvalue 9 with eigenvector (2q1+q2)/5(2q_1+q_2)/\sqrt{5}. Choice A assumes the perturbation direction uu itself is an eigenvector, but rank-1 updates to matrices with unequal existing eigenvalues generally rotate the eigenvectors. Choice C guesses q1q_1 remains an eigenvector, which fails once off-diagonal coupling is introduced. Choice D picks the correct functional form but wrong coefficients — the eigenvector for λ=4\lambda=4 is (q1−2q2)/5(q_1-2q_2)/\sqrt{5}, not for the largest eigenvalue. When you see a rank-1 update, always restrict the problem to the affected subspace and solve a small eigenvalue problem there — this avoids dimension-bloat and reveals exactly how eigenvectors rotate.

Question 10

An analyst observes n=5n=5 independent realizations of a continuous p=8p=8 dimensional distribution. The observations are in general affine position, and the usual sample covariance matrix is computed after subtracting the sample mean.

How many zero eigenvalues does the sample covariance matrix have?

  1. It has exactly 33 zero eigenvalues because the uncentered data matrix has rank 55.
  2. It has exactly 44 zero eigenvalues because centering leaves rank 44. (correct answer)
  3. It has exactly 55 zero eigenvalues because the covariance divisor is n−1n-1.
  4. It has no zero eigenvalues because the population distribution is continuous.
Explanation: Whenever you see a question about eigenvalues of a sample covariance matrix, your first instinct should be to think about rank, not about the divisor or the population's properties. Here's the key chain of reasoning. You start with n=5n = 5 observations in p=8p = 8 dimensions. Stack these into a data matrix XX of size 5×85 \times 8. Before centering, this matrix has rank at most min⁡(5,8)=5\min(5, 8) = 5. When you subtract the sample mean, you project the rows onto a subspace orthogonal to the all-ones vector, which reduces the row rank by exactly 1. So the centered data matrix has rank at most 44. The sample covariance matrix S=1n−1Xc⊤XcS = \frac{1}{n-1} X_c^\top X_c, where XcX_c is the centered matrix, is an 8×88 \times 8 matrix with rank at most 44. By the rank-nullity theorem, a rank-4 matrix in R8\mathbb{R}^8 has a null space of dimension 8−4=48 - 4 = 4, meaning exactly 4 zero eigenvalues. This confirms answer B. Choice A incorrectly uses the rank of the uncentered data matrix (rank 5), forgetting that centering costs one additional rank. Choice C confuses the n−1n-1 denominator — that's just a scalar rescaling and has absolutely no effect on the number of zero eigenvalues. Choice D is a tempting trap: continuity of the population distribution guarantees the population covariance matrix is positive definite, but the sample covariance matrix is constrained by the geometry of finitely many observations, not the population. Remember: for sample covariance matrices, always think rank=min⁡(n−1,p)\text{rank} = \min(n-1, p), and zero eigenvalues =p−rank= p - \text{rank}.