Diagonalization - Linear Algebra

SECTION 1

Historical Context & Motivation

Imagine you have a complicated machine with dozens of gears and levers. What if you could take it apart, see what each piece does on its own, and then put it back together? That is essentially what diagonalization does for matrices. It breaks a matrix into simpler pieces so we can understand — and compute with — it much more easily.

The story of diagonalization stretches back centuries. Mathematicians needed ways to solve large systems of equations, understand vibrations in bridges and buildings, and compute things that would otherwise take enormous effort. Over time, the ideas of eigenvalues (special scaling numbers) and eigenvectors (special directions) came together to form the technique we now call diagonalization.

1750s • Euler & Vibrating Strings

Leonhard Euler studied vibrating strings and found that certain characteristic values (what we now call eigenvalues) naturally appeared when describing how strings oscillate.

1826 • Cauchy's Matrix Foundations

Augustin-Louis Cauchy proved key results about the eigenvalues of symmetric matrices, laying the groundwork for the diagonalization theorem.

1855 • Cayley Invents Matrix Algebra

Arthur Cayley formally defined matrix multiplication and introduced the idea of treating arrays of numbers as single mathematical objects that can be added, multiplied, and decomposed.

1904 • Hilbert's Spectral Theory

David Hilbert extended eigenvalue ideas to infinite dimensions, connecting diagonalization to quantum mechanics and modern physics.

1950s+ • Computer-Age Applications

With the rise of computers, diagonalization became essential for graphics, data science, engineering simulations, and Google's PageRank algorithm.

The central question diagonalization answers is: Can we find a special coordinate system where a matrix just stretches things along the axes, with no rotation or mixing? If the answer is yes, calculations like raising a matrix to the 100th power become almost as easy as multiplying regular numbers.

SECTION 2

Core Principles & Definitions

Before diving into diagonalization, let's make sure you're comfortable with a few key ideas. A matrix is a rectangular grid of numbers. When we multiply a matrix by a vector (a column of numbers), the vector usually changes both its direction and its length. However, some special vectors only get stretched or flipped — they keep pointing in the same (or exactly opposite) direction. These are eigenvectors.

Eigenvector

A nonzero vector v that, when multiplied by matrix A, comes out pointing in the same direction: Av = λv. Think of it as an arrow the matrix doesn't rotate.

Eigenvalue (λ)

The number λ (lambda) that tells you how much the eigenvector gets stretched or shrunk. If λ = 2, the vector doubles in length. If λ = −1, it flips direction.

Diagonal Matrix

A matrix where only the entries on the main diagonal (top-left to bottom-right) can be nonzero. Everything else is 0. These are the simplest matrices to work with.

Diagonalizable

A matrix is diagonalizable if we can find enough eigenvectors to form a complete set (a basis). Then we can rewrite the matrix in a simpler diagonal form.

The Factorization A = PDP⁻¹

When a matrix is diagonalizable, we can write it as A = PDP⁻¹, where P holds the eigenvectors as columns, D is the diagonal matrix of eigenvalues, and P⁻¹ undoes the change of coordinates.

✦

KEY TAKEAWAY

Think of diagonalization like switching a TV to a better camera angle during a sports game. The game (the matrix) hasn't changed, but from this new angle (the eigenvector basis), everything looks simpler — each player just runs straight up or down the field. The eigenvalues are the speeds of each player, and the eigenvectors are the lanes they run in.

SECTION 3

Visual Explanation

The diagram below shows what happens when a 2×2 matrix acts on different vectors. Most vectors change both direction and length (shown in gray). But the eigenvectors (shown in color) only get stretched — they stay on the same line through the origin.

On the left, we see several vectors before the matrix acts on them. On the right, the same vectors after multiplication by matrix A. The gray vectors changed direction, but the cyan eigenvector v₁ (eigenvalue 3) and pink eigenvector v₂ (eigenvalue 1.5) simply got stretched along their original lines.

This is the heart of diagonalization. When we switch to the eigenvector coordinate system, the matrix becomes a diagonal matrix — it just stretches each axis by the corresponding eigenvalue. No mixing, no rotation, just simple scaling in each direction.

SECTION 4

Mathematical Framework

Let's build the math step by step. We start with the eigenvalue equation and work our way up to the full diagonalization formula.

EIGENVALUE EQUATION

Av = λv

A = the square matrix we want to diagonalize; v = an eigenvector (nonzero); λ = the eigenvalue (a number). This says 'multiplying v by A has the same effect as multiplying v by the number λ.'

CHARACTERISTIC EQUATION

det(A − λI) = 0

To find eigenvalues, we solve this equation. I is the identity matrix (1s on the diagonal, 0s elsewhere). det means determinant. Solving this gives us the eigenvalues λ₁, λ₂, etc.

DIAGONALIZATION FORMULA

A = PDP⁻¹

P = the matrix whose columns are the eigenvectors of A; D = the diagonal matrix with eigenvalues on the diagonal; P⁻¹ = the inverse of P. This is the big payoff: A is 'equivalent' to the simple diagonal matrix D.

MATRIX POWER SHORTCUT

Aⁿ = PDⁿP⁻¹

Since D is diagonal, raising it to the nth power is easy — just raise each diagonal entry to the nth power. This makes computing A¹⁰⁰ as simple as raising a few numbers to the 100th power!

💡

When Can We Diagonalize?

An n × n matrix is diagonalizable when it has n linearly independent eigenvectors. A quick rule: if an n × n matrix has n distinct (different) eigenvalues, it is always diagonalizable. Repeated eigenvalues might still work, but you need to check if there are enough eigenvectors.

SECTION 5

Step-by-Step Process

Diagonalizing a matrix follows a clear recipe. The diagram below shows the complete process from start to finish. Let's walk through each stage.

The complete diagonalization workflow. Starting from matrix A, we find eigenvalues (Step 2), eigenvectors (Step 3), check that we have enough (Step 4), and then assemble the matrices P and D (Step 5). The final result is the factorization A = PDP⁻¹.

Let's unpack the most important step — finding eigenvalues. For a 2×2 matrix, the characteristic equation turns into a quadratic that you can solve with the quadratic formula. For a 3×3 matrix, it becomes a cubic equation. In practice, larger matrices are handled by computer algorithms.

Once you have the eigenvalues, you plug each one back into (A − λI)v = 0 and solve for the eigenvector v. This is a system of linear equations — something you may have practiced with substitution or elimination.

SECTION 6

Worked Example

Let's diagonalize the 2×2 matrix A = [[4, 1], [2, 3]]. We'll follow every step of the process.

Diagonalize A = [[4, 1], [2, 3]]

1
Step 1 — Set Up the Characteristic Equation

We need det(A − λI) = 0. First, compute A − λI: A − λI = [[4 − λ, 1], [2, 3 − λ]] The determinant of a 2×2 matrix [[a, b], [c, d]] is ad − bc. So: det(A − λI) = (4 − λ)(3 − λ) − (1)(2)
2
Step 2 — Solve for Eigenvalues

Expand: (4 − λ)(3 − λ) = 12 − 4λ − 3λ + λ² = λ² − 7λ + 12 Subtract 2: λ² − 7λ + 10 = 0 Factor: (λ − 5)(λ − 2) = 0

λ₁ = 5, λ₂ = 2
3
Step 3a — Find Eigenvector for λ₁ = 5

Plug λ = 5 into (A − λI)v = 0: A − 5I = [[−1, 1], [2, −2]] Row 1 gives: −x₁ + x₂ = 0, so x₂ = x₁. Choose x₁ = 1.

v₁ = [1, 1]
4
Step 3b — Find Eigenvector for λ₂ = 2

Plug λ = 2 into (A − λI)v = 0: A − 2I = [[2, 1], [2, 1]] Row 1 gives: 2x₁ + x₂ = 0, so x₂ = −2x₁. Choose x₁ = 1.

v₂ = [1, −2]
5
Step 4 — Build P and D

We have 2 independent eigenvectors for a 2×2 matrix — success! Place them as columns of P: P = [[1, 1], [1, −2]] D = [[5, 0], [0, 2]]

A = PDP⁻¹ = [[1, 1], [1, −2]] · [[5, 0], [0, 2]] · P⁻¹
6
Step 5 — Verify (Optional but Satisfying!)

Let's check: A·v₁ = [[4,1],[2,3]]·[1,1] = [4+1, 2+3] = [5, 5] = 5·[1,1] ✓ A·v₂ = [[4,1],[2,3]]·[1,−2] = [4−2, 2−6] = [2, −4] = 2·[1,−2] ✓ Both eigenvector equations check out perfectly.

Diagonalization confirmed! ✓

SECTION 7

Strengths & Limitations

Diagonalization is a powerful tool, but like any tool, it has its best uses and its limitations. Let's compare what it does well with where it falls short.

Aspect	Strength	Limitation
Matrix Powers	Computing Aⁿ becomes trivial — just raise each eigenvalue to the nth power.	Only works if the matrix is diagonalizable in the first place.
Understanding Structure	Reveals the fundamental 'directions' (eigenvectors) and 'rates' (eigenvalues) of a transformation.	Some matrices have complex (imaginary) eigenvalues, making interpretation harder for beginners.
Applicability	All symmetric matrices (like those in physics) are always diagonalizable.	Not all matrices can be diagonalized. For example, [[0, 1], [0, 0]] has only one eigenvector.
Computation Cost	After diagonalizing once, many future calculations become much faster.	Finding eigenvalues of very large matrices can be computationally expensive.

Strengths and limitations of diagonalization

✦

KEY TAKEAWAY

Think of diagonalization like learning a shortcut route to school. It takes some effort to discover the shortcut (finding eigenvalues and eigenvectors), but once you know it, your daily commute (matrix computations) is dramatically faster. However, not every neighborhood has a shortcut — some matrices simply aren't diagonalizable, and you need a different technique (like Jordan Normal Form) for those.

SECTION 8

Connection to Advanced Topics

Diagonalization is a stepping stone to many advanced ideas in mathematics and science. Understanding how it connects to more powerful techniques helps you see why it's such an important foundational skill.

Diagonalization (This Lesson)	Advanced Extension
A = PDP⁻¹ using eigenvectors	Singular Value Decomposition (SVD) — works for ANY matrix, not just square ones. Used in data compression and machine learning.
Not diagonalizable → stuck	Jordan Normal Form — an 'almost diagonal' form that works for every square matrix, even non-diagonalizable ones.
Eigenvalues are real numbers	Spectral Theory — extends eigenvalue ideas to infinite-dimensional spaces (used in quantum mechanics).
Aⁿ = PDⁿP⁻¹ for integer n	Matrix Exponentials — $e^{At}$ = $Pe^{Dt}$ P⁻¹ solves systems of differential equations.

How diagonalization connects to advanced topics

In the real world, diagonalization powers technologies you use every day. Google's PageRank algorithm uses eigenvalues to rank web pages. Facial recognition software uses a technique called Principal Component Analysis (PCA), which is built directly on diagonalization. And in physics, the vibrations of molecules and the behavior of quantum particles are both described by diagonalizing matrices.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

In your own words, explain what it means for a matrix to be diagonalizable. Why would we want to diagonalize a matrix instead of just using it directly?

Show answer

A matrix is diagonalizable when we can find a set of special directions (eigenvectors) where the matrix acts as simple scaling. We write this as A = PDP⁻¹, where D is a diagonal matrix of eigenvalues. We want to do this because diagonal matrices are much easier to compute with — raising D to a power just means raising each diagonal entry to that power. This makes calculations like A¹⁰⁰ manageable, which would otherwise require multiplying the matrix by itself 100 times.

Question 2: BASIC CALCULATION

Find the eigenvalues of the matrix A = [[3, 0], [0, 7]]. Is this matrix already diagonal? What are its eigenvectors?

Show answer

Yes, A is already diagonal! For a diagonal matrix, the eigenvalues are simply the entries on the diagonal: λ₁ = 3 and λ₂ = 7. The eigenvectors are the standard basis vectors: v₁ = [1, 0] (for λ₁ = 3) and v₂ = [0, 1] (for λ₂ = 7). You can verify: A·[1,0] = [3,0] = 3·[1,0] and A·[0,1] = [0,7] = 7·[0,1]. In this case, P is the identity matrix and D = A itself.

Question 3: INTERMEDIATE

Diagonalize the matrix A = [[5, 4], [1, 2]]. Find the eigenvalues, eigenvectors, and write A = PDP⁻¹.

Show answer

First, find eigenvalues using det(A − λI) = 0: det([[5−λ, 4], [1, 2−λ]]) = (5−λ)(2−λ) − 4 = λ² − 7λ + 6 = (λ−6)(λ−1) = 0. So λ₁ = 6 and λ₂ = 1. For λ₁ = 6: (A − 6I)v = 0 gives [[−1,4],[1,−4]]v = 0, so x₂ = x₁/4; choose v₁ = [4, 1]. For λ₂ = 1: (A − I)v = 0 gives [[4,4],[1,1]]v = 0, so x₂ = −x₁; choose v₂ = [1, −1]. Therefore P = [[4, 1], [1, −1]] and D = [[6, 0], [0, 1]], giving A = PDP⁻¹.

Question 4: APPLIED

A population model uses the matrix A = [[0.5, 0.3], [0.5, 0.7]] to predict next year's population distribution. If the initial population vector is p₀ = [1000, 2000], use diagonalization to find the population after 2 steps (p₂ = A²p₀). Hint: First diagonalize A, then use A² = PD²P⁻¹.

Show answer

The eigenvalues of A are found from det(A − λI) = (0.5−λ)(0.7−λ) − 0.15 = λ² − 1.2λ + 0.2 = (λ−1)(λ−0.2) = 0, giving λ₁ = 1 and λ₂ = 0.2. For λ₁ = 1: v₁ = [3, 5] (from −0.5x₁ + 0.3x₂ = 0). For λ₂ = 0.2: v₂ = [1, −1] (from 0.3x₁ + 0.3x₂ = 0). So D² = [[1, 0], [0, 0.04]]. Computing A²p₀ = PD²P⁻¹p₀, the population stabilizes toward the eigenvector for λ = 1, which represents the long-run steady state. After working through the matrix multiplication, p₂ ≈ [1137.5, 1862.5] (rounding may vary depending on method).

Question 5: CRITICAL THINKING

Consider the matrix B = [[2, 1], [0, 2]]. Try to diagonalize it. What happens? Explain why some matrices cannot be diagonalized, and describe what makes this matrix different from the ones we successfully diagonalized.

Show answer

The characteristic equation is det(B − λI) = (2−λ)² = 0, giving a repeated eigenvalue λ = 2. For λ = 2: (B − 2I)v = [[0,1],[0,0]]v = 0 gives x₂ = 0, but x₁ is free. So the only eigenvector direction is v = [1, 0]. We need 2 independent eigenvectors for a 2×2 matrix, but we only found 1. Therefore B is NOT diagonalizable. The key difference is that B has a repeated eigenvalue that doesn't produce enough eigenvectors. Matrices with all distinct eigenvalues are always diagonalizable. Repeated eigenvalues are the 'danger zone' — they might or might not produce enough eigenvectors. Matrix B would require Jordan Normal Form instead.

SUMMARY

Lesson Summary

Summary

Diagonalization is a technique that rewrites a square matrix as A = PDP⁻¹, where D is a simple diagonal matrix of eigenvalues and P is a matrix whose columns are the corresponding eigenvectors. The process involves solving the characteristic equation det(A − λI) = 0 to find eigenvalues, then finding eigenvectors for each eigenvalue.

A matrix is diagonalizable when it has n linearly independent eigenvectors (for an n × n matrix). The biggest payoff is that matrix powers become easy: Aⁿ = PDⁿP⁻¹ reduces to simply raising each eigenvalue to the nth power. This idea underpins applications in population modeling, computer graphics, data science, and quantum physics.