Singular Value Decomposition (SVD) - Linear Algebra

  1. What does the largest singular value σ1\sigma_1 of AA measure?
    The maximum of ∥Ax∥\|A\mathbf{x}\| over unit vectors x\mathbf{x}, reached at x=v1\mathbf{x} = \mathbf{v}_1.
  2. A symmetric matrix has eigenvalues 44 and −7-7. What are its singular values?
    77 and 44. For a symmetric matrix the singular values are the absolute values of the eigenvalues.
  3. What are the columns of VV in the SVD A=UΣVTA = U\Sigma V^T?
    Orthonormal eigenvectors of ATAA^TA, called the right singular vectors.
  4. What are the singular values of an n×nn \times n orthogonal matrix?
    All equal to 11, since QTQ=IQ^TQ = I has every eigenvalue equal to 11.
  5. How are the singular values of AA obtained from ATAA^TA?
    They are the square roots of the eigenvalues of ATAA^TA, which are all nonnegative.
  6. In the SVD A=UΣVTA = U\Sigma V^T of an m×nm \times n matrix, what are UU, Σ\Sigma and VV?
    UU is m×mm \times m orthogonal, Σ\Sigma is m×nm \times n with nonnegative singular values on its diagonal, and VV is n×nn \times n orthogonal.
  7. In the full SVD of a 2×32 \times 3 matrix, what is the size of Σ\Sigma?
    2×32 \times 3, the same size as AA.
  8. ATAA^TA has eigenvalues 1616, 99 and 00. What are the singular values of AA?
    44, 33 and 00, the square roots of the eigenvalues.
  9. How is the best rank-kk approximation of AA formed from its SVD?
    Keep the kk largest singular values: Ak=σ1u1v1T+⋯+σkukvkTA_k = \sigma_1\mathbf{u}_1\mathbf{v}_1^T + \dots + \sigma_k\mathbf{u}_k\mathbf{v}_k^T.
  10. How does the number of nonzero singular values of AA relate to AA?
    It equals the rank of AA.
  11. For a nonzero singular value σi\sigma_i with right singular vector vi\mathbf{v}_i, how is the left singular vector ui\mathbf{u}_i found?
    ui=1σiAvi\mathbf{u}_i = \frac{1}{\sigma_i}A\mathbf{v}_i. These vectors are orthonormal eigenvectors of AATAA^T.
  12. Which real matrices have a singular value decomposition?
    Every real matrix, of any size and any rank. Unlike diagonalization, no condition is needed.

Study these flashcards