01 / 20

Orthogonal
Diagonalization

Learn how symmetric matrices can be decomposed using orthogonal eigenvectors into the elegant factorization \( A = PDP^T \).

Linear Algebra

What is Orthogonal Diagonalization?

Orthogonal diagonalization is the process of factoring a matrix \( A \) as:

$$ A = PDP^T $$

where \( P \) is an orthogonal matrix (its columns are orthonormal eigenvectors) and \( D \) is a diagonal matrix of eigenvalues.

Key Definitions

Symmetric Matrix
A matrix \( A \) where \( A = A^T \). Every entry satisfies \( a_{ij} = a_{ji} \).
Orthogonal Matrix
A matrix \( P \) where \( P^T P = I \). Its columns form an orthonormal set.
Diagonal Matrix
A matrix \( D \) with eigenvalues \( \lambda_1, \lambda_2, \ldots \) on the main diagonal and zeros elsewhere.

Why Does It Matter?

Orthogonal diagonalization reveals the principal axes of a transformation. It turns complex matrix operations into simple scaling along perpendicular directions.

Computing Powers
\( A^n = PD^nP^T \), making \( A^{100} \) trivial to compute.
Quadratic Forms
Classify \( \mathbf{x}^T A \mathbf{x} \) as positive/negative definite using eigenvalues.
PCA / Data Science
Principal Component Analysis diagonalizes the covariance matrix to find directions of maximum variance.

The Spectral Theorem

The Spectral Theorem guarantees two powerful properties for any real symmetric matrix:

1
All eigenvalues are real. No complex eigenvalues ever appear for symmetric matrices.
2
Eigenvectors from distinct eigenvalues are orthogonal. They naturally form perpendicular directions.
3
Eigenspaces have full dimension. There are always enough independent eigenvectors to form a basis.

Visualizing \( A = PDP^T \)

A symmetric = P orthogonal · D diagonal ·
\( P^T \)
\( = P^{-1} \)
 eigenvectors as columns eigenvalues on diagonal

Diagonalizable vs Orthogonally Diagonalizable

Diagonalizable Orthogonally Diag.
Factorization \( A = PDP^{-1} \) \( A = PDP^T \)
Matrix \( P \) Any invertible matrix Orthogonal: \( P^TP = I \)
Requirement \( n \) linearly independent eigenvectors \( A \) must be symmetric
Eigenvalues May be complex Always real
Inverse of \( P \) Must compute \( P^{-1} \) Free: just transpose

How to Orthogonally Diagonalize

1
Verify symmetry. Check that \( A = A^T \). If not symmetric, stop: orthogonal diagonalization is impossible.
2
Find eigenvalues. Solve \( \det(A - \lambda I) = 0 \) for all \( \lambda \).
3
Find eigenvectors. For each \( \lambda \), solve \( (A - \lambda I)\mathbf{x} = \mathbf{0} \) to get the eigenspace basis.
4
Orthogonalize within each eigenspace. If an eigenspace has dimension > 1, apply the Gram-Schmidt process to its basis vectors.
5
Normalize all eigenvectors. Divide each by its length to get unit vectors.
6
Build \( P \) and \( D \). Columns of \( P \) are the orthonormal eigenvectors. \( D \) has matching eigenvalues on the diagonal.

Example 1: \( 2 \times 2 \) Matrix

Orthogonally diagonalize \( A = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix} \).

Example 1: Result

The orthogonal diagonalization of \( A \) is:

$$ A = PDP^T = \frac{1}{\sqrt{2}}\begin{pmatrix} -1 & 1 \\ 1 & 1 \end{pmatrix} \begin{pmatrix} 2 & 0 \\ 0 & 4 \end{pmatrix} \frac{1}{\sqrt{2}}\begin{pmatrix} -1 & 1 \\ 1 & 1 \end{pmatrix} $$

\( \mathbf{u}_1 \)
\( \mathbf{u}_2 \)
\( 2\mathbf{u}_1 \)
\( 4\mathbf{u}_2 \)
 Before: unit circle After: scaled by eigenvalues A

Example 2: \( 3 \times 3 \) Matrix

Orthogonally diagonalize \( A = \begin{pmatrix} 2 & 1 & 1 \\ 1 & 2 & 1 \\ 1 & 1 & 2 \end{pmatrix} \).

Example 2: Gram-Schmidt Step

The eigenspace for \( \lambda = 1 \) has two basis vectors. We apply Gram-Schmidt to make them orthogonal:

1
Set \( \mathbf{w}_1 = \begin{pmatrix} -1 \\ 1 \\ 0 \end{pmatrix} \).
2
Compute \( \mathbf{w}_2 = \mathbf{v}_2 - \frac{\mathbf{v}_2 \cdot \mathbf{w}_1}{\mathbf{w}_1 \cdot \mathbf{w}_1}\mathbf{w}_1 = \begin{pmatrix} -1 \\ 0 \\ 1 \end{pmatrix} - \frac{1}{2}\begin{pmatrix} -1 \\ 1 \\ 0 \end{pmatrix} = \begin{pmatrix} -1/2 \\ -1/2 \\ 1 \end{pmatrix} \).
3
Normalize all three eigenvectors to unit length to build \( P \).

Example 3: Using the Decomposition

Given \( A = PDP^T \) with \( D = \begin{pmatrix} 2 & 0 \\ 0 & 5 \end{pmatrix} \), compute \( A^3 \).

When is Gram-Schmidt Needed?

The Spectral Theorem guarantees eigenvectors from different eigenvalues are already orthogonal. But when an eigenvalue has algebraic multiplicity > 1, the basis vectors within that eigenspace may not be orthogonal.

Distinct eigenvalues
Each eigenspace is 1-dimensional. Eigenvectors are automatically orthogonal. Just normalize.
Repeated eigenvalue
Eigenspace dimension > 1. Must apply Gram-Schmidt within that eigenspace, then normalize.

Common Mistakes

✕ Forgetting to Normalize
Building \( P \) from eigenvectors that are orthogonal but not unit length. \( P^T P \neq I \) unless every column has length 1.
✕ Skipping Gram-Schmidt
When a repeated eigenvalue gives a 2D eigenspace, the natural basis vectors may not be orthogonal. Always check and orthogonalize if needed.
✕ Non-Symmetric Matrix
Attempting orthogonal diagonalization on a non-symmetric matrix. It may be diagonalizable but not orthogonally diagonalizable.

Summary: Key Properties

Property Symmetric \( A \) General \( A \)
Eigenvalues Always real May be complex
Eigenvectors (distinct \( \lambda \)) Orthogonal Linearly independent
Always diagonalizable? Yes Not guaranteed
Factorization \( PDP^T \) \( PDP^{-1} \)
Computing \( P^{-1} \) Free (just transpose) Row reduction needed
Application Quadratic forms, PCA General lin. systems

Question 1 of 5

Which condition is necessary and sufficient for a real matrix \( A \) to be orthogonally diagonalizable?

Question 2 of 5

In the factorization \( A = PDP^T \), what is special about the matrix \( P \)?

Question 3 of 5

If \( A = PDP^T \) and \( D = \begin{pmatrix} 3 & 0 \\ 0 & 7 \end{pmatrix} \), what is \( A^4 \)?

Question 4 of 5

A symmetric \( 3 \times 3 \) matrix has eigenvalues \( \lambda = 2 \) (multiplicity 2) and \( \lambda = 5 \) (multiplicity 1). When must you apply Gram-Schmidt?

Question 5 of 5

Which matrix is not orthogonally diagonalizable?