Principal Directions & Quadratic Forms — Principal Directions and Quadratic Forms (Intro) - Linear Algebra

SECTION 1

Historical Context & Motivation

Imagine you grab a rubber sheet and stretch it. Most points on the sheet move in complicated ways — sliding sideways while also being pulled outward. But a few special directions stay perfectly in line with the pull. These are called principal directions, and mathematicians have been studying them for centuries.

The story begins with curves and surfaces. Early mathematicians wanted to understand the shapes described by equations like x² + 3xy + y² = 1. These equations are called quadratic forms because every term contains variables raised to the second power. The challenge was figuring out whether such an equation describes a circle, an ellipse, a hyperbola, or something else entirely.

1748 • Euler Studies Rotating Axes

Leonhard Euler showed that rotating the coordinate axes by the right angle can eliminate the messy cross-terms (like xy) from a quadratic equation, revealing the true shape of the curve.

1829 • Cauchy Proves the Spectral Theorem

Augustin-Louis Cauchy proved that every symmetric matrix has real eigenvalues and perpendicular eigenvectors. This guaranteed that principal directions always exist for quadratic forms.

1855 • Sylvester Classifies Quadratic Forms

James Joseph Sylvester developed the Law of Inertia, which classifies quadratic forms by counting positive, negative, and zero eigenvalues — a tool still used today.

1900s • Modern Applications Emerge

Principal directions became essential in engineering (stress analysis), physics (moments of inertia), statistics (principal component analysis), and computer graphics (shape recognition).

The central question these mathematicians were asking is still the question we tackle today: given a quadratic form, can we find a new set of axes that reveals the simplest possible version of the equation? The answer is yes — and eigenvectors point us to exactly those axes.

SECTION 2

Core Principles & Definitions

Before we dive into the math, let's nail down the key ideas. Think of a matrix as a machine that takes in an arrow (a vector) and spits out a new arrow. Usually the output arrow points in a totally different direction. But for certain special input arrows, the machine only stretches or shrinks them — it doesn't rotate them at all. Those special arrows are the principal directions.

Quadratic Form

An expression like Q(x, y) = ax² + 2bxy + cy² where every term has degree two. It can be written using a matrix: Q = xᵀAx, where A is a symmetric matrix.

Symmetric Matrix

A square matrix where the entry in row i, column j equals the entry in row j, column i. In other words, A = Aᵀ. Quadratic forms always use symmetric matrices.

Principal Direction (Eigenvector)

A nonzero vector v such that Av = λv. The matrix only scales this vector; it doesn't change its direction. The scalar λ is the eigenvalue.

Eigenvalue (λ)

The stretch factor along a principal direction. A positive λ means the vector keeps its direction; a negative λ means it flips. Zero means the vector collapses to the origin.

Diagonalization

The process of rewriting a quadratic form using principal directions as new axes. This eliminates cross-terms (like xy), leaving only squared terms like λ₁u² + λ₂v².

✦

KEY TAKEAWAY

Think of a quadratic form like a tilted oval drawn on graph paper. The principal directions are the natural axes of the oval — the long axis and the short axis. When you rotate your graph paper to line up with those axes, the equation of the oval becomes much simpler, with no xy terms. It's like turning a crooked picture frame until it's perfectly level.

SECTION 3

Seeing Principal Directions in Action

The diagram below shows how a quadratic form Q(x, y) = 5x² + 4xy + 2y² creates an ellipse when set equal to a constant. The original x- and y-axes don't line up with the ellipse. But the principal directions (the eigenvectors of the associated matrix) point exactly along the long and short axes of the ellipse.

The violet ellipse is the set of all (x, y) where Q = 10. The cyan dashed line shows the first principal direction (eigenvector v₁), aligned with the long axis. The pink dashed line shows the second principal direction (eigenvector v₂), aligned with the short axis. Notice they are perpendicular to each other.

In the diagram, the ellipse doesn't line up with the regular x- and y-axes — it's tilted. That tilt comes from the 4xy cross-term in the equation. When we switch to the principal directions as our new axes, the cross-term disappears, and the equation becomes simply λ₁u² + λ₂v² = 10, where u and v are coordinates along the new axes. That's just a standard, un-tilted ellipse!

SECTION 4

Mathematical Framework

Let's see how the math connects matrices, eigenvalues, and quadratic forms. We'll work in two dimensions to keep things clear, but the same ideas extend to any number of dimensions.

From Quadratic Form to Matrix

QUADRATIC FORM IN MATRIX NOTATION

Q(x, y) = ax² + 2bxy + cy² = [x y] · [[a, b], [b, c]] · [[x], [y]] = xᵀAx

Here A is a 2×2 symmetric matrix with entries a, b on the diagonal corners and b in the off-diagonal spots. The factor of 2 in front of bxy splits evenly between the two off-diagonal entries.

Finding Principal Directions

EIGENVALUE EQUATION

Av = λv

A principal direction v satisfies this equation. The matrix A acting on v produces the same vector v, just scaled by λ. To find λ, we solve det(A − λI) = 0, called the characteristic equation.

CHARACTERISTIC EQUATION (2×2)

det([[a − λ, b], [b, c − λ]]) = (a − λ)(c − λ) − b² = 0

This is a quadratic equation in λ. Solving it gives two eigenvalues λ₁ and λ₂. For a symmetric matrix, both eigenvalues are always real numbers (never complex), which is a wonderful guarantee.

DIAGONALIZED QUADRATIC FORM

Q = λ₁u² + λ₂v²

After rotating to the principal directions, the quadratic form has no cross-term. The variables u and v are the coordinates along the eigenvectors. The eigenvalues λ₁ and λ₂ tell us the stretch in each principal direction.

ℹ

Why Symmetric Matrices?

You might wonder: why must A be symmetric? It's because any quadratic form Q(x, y) = ax² + 2bxy + cy² can always be represented by a symmetric matrix. If someone hands you a non-symmetric matrix, you can always replace it with the symmetric version (A + Aᵀ)/2 without changing the quadratic form. Symmetric matrices have the bonus property that their eigenvectors are always perpendicular to each other.

SECTION 5

Classifying Quadratic Forms by Eigenvalues

Once you find the eigenvalues of a quadratic form's matrix, you can immediately classify the shape of the curve (in 2D) or surface (in 3D). The sign of each eigenvalue — positive, negative, or zero — tells the whole story.

Five classifications of 2D quadratic forms based on eigenvalue signs. Positive definite forms produce ellipses that never touch the origin. Indefinite forms produce hyperbolas. The semi-definite cases are borderline — one eigenvalue is zero, so the curve collapses into parallel lines.

Classification	Eigenvalue Signs	Shape (2D)	Example Equation
Positive Definite	λ₁ > 0, λ₂ > 0	Ellipse	3x² + 2y² = 6
Negative Definite	λ₁ < 0, λ₂ < 0	Ellipse (Q < 0)	−x² − 4y² = −5
Indefinite	λ₁ > 0, λ₂ < 0	Hyperbola	x² − y² = 1
Pos. Semi-Definite	λ₁ > 0, λ₂ = 0	Parallel lines or point	x² = 4
Neg. Semi-Definite	λ₁ < 0, λ₂ = 0	Parallel lines or point	−y² = −1

Classification of 2D quadratic forms

SECTION 6

Worked Example: Finding Principal Directions

Let's work through a complete example. We'll start with the quadratic form Q(x, y) = 5x² + 4xy + 2y², find its matrix, compute the eigenvalues (stretch factors), find the eigenvectors (principal directions), and rewrite the quadratic form in its simplified diagonal version.

Diagonalizing Q(x, y) = 5x² + 4xy + 2y²

1
Step 1 — Write the Symmetric Matrix

Compare Q = 5x² + 4xy + 2y² with the general form ax² + 2bxy + cy². We see a = 5, 2b = 4 so b = 2, and c = 2. The matrix is A = [[5, 2], [2, 2]]. Notice it's symmetric: the off-diagonal entries are both 2.

A = [[5, 2], [2, 2]]
2
Step 2 — Set Up the Characteristic Equation

We need det(A − λI) = 0. That means det([[5 − λ, 2], [2, 2 − λ]]) = 0. Expanding: (5 − λ)(2 − λ) − (2)(2) = 0. Distribute: 10 − 5λ − 2λ + λ² − 4 = 0. Simplify: λ² − 7λ + 6 = 0.

λ² − 7λ + 6 = 0
3
Step 3 — Solve for the Eigenvalues

Factor the quadratic: (λ − 1)(λ − 6) = 0. So the two eigenvalues are λ₁ = 1 and λ₂ = 6. Both are positive, which tells us this quadratic form is positive definite — the curve is an ellipse.

λ₁ = 1, λ₂ = 6
4
Step 4 — Find the Eigenvectors (Principal Directions)

For λ₁ = 1: Solve (A − I)v = 0 → [[4, 2], [2, 1]]v = 0. The first row says 4v₁ + 2v₂ = 0, so v₂ = −2v₁. Choose v₁ = 1: v₁ = [1, −2]. For λ₂ = 6: Solve (A − 6I)v = 0 → [[−1, 2], [2, −4]]v = 0. The first row says −v₁ + 2v₂ = 0, so v₁ = 2v₂. Choose v₂ = 1: v₂ = [2, 1]. Check: v₁ · v₂ = (1)(2) + (−2)(1) = 0 ✓ They are perpendicular!

v₁ = [1, −2] for λ₁ = 1; v₂ = [2, 1] for λ₂ = 6
5
Step 5 — Write the Diagonalized Form

Using coordinates u and v along the principal directions, the quadratic form becomes Q = 1 × u² + 6 × v² = u² + 6v². No cross-term! Setting Q = 10, we get u²/10 + v²/(10/6) = 1, which is a standard ellipse with semi-axes √10 and √(10/6).

Q = u² + 6v² (diagonal form, no cross-term)

SECTION 7

Strengths & Limitations of the Principal Directions Approach

Diagonalizing quadratic forms through principal directions is a powerful technique, but it's important to understand when it works beautifully and where you might run into limits.

Strengths	Limitations
Always works for symmetric matrices — guaranteed real eigenvalues and perpendicular eigenvectors.	Only applies directly to quadratic (degree-2) expressions. Cubic or higher-degree forms need different tools.
Eliminates cross-terms completely, making the equation easy to read and classify.	For large matrices (say 100×100), computing eigenvalues by hand is impractical — you need a computer.
Reveals the geometry immediately: eigenvalue signs tell you the shape (ellipse, hyperbola, etc.).	If the matrix is not symmetric, the eigenvectors may not be perpendicular, and the technique doesn't apply in the same clean way.
Widely applicable across physics, engineering, statistics, and computer science.	Repeated eigenvalues (like λ₁ = λ₂ = 3) mean any direction is a principal direction — the shape is a circle, so there's no single 'special' axis.

Strengths vs. Limitations of Principal Directions

🔑

BIG PICTURE

Principal directions are like finding the handle on a suitcase. The suitcase (quadratic form) can be awkwardly shaped, but once you grab the handle (the eigenvectors), carrying it becomes much easier. In more advanced courses, this same idea powers Principal Component Analysis (PCA) in data science and stress tensor analysis in engineering.

SECTION 8

Connection to Advanced Theory

What you've learned here is the 2D introductory version of a much bigger story. The same ideas extend to higher dimensions, and they show up in some surprising places. Here's a quick comparison of what we covered versus what comes next.

What We Learned (Intro)	Where It Goes Next (Advanced)
2×2 symmetric matrices with two eigenvalues	n×n symmetric matrices with n eigenvalues — the Spectral Theorem
Classifying ellipses and hyperbolas in 2D	Classifying quadric surfaces (ellipsoids, hyperboloids, paraboloids) in 3D and beyond
Rotating axes to remove one cross-term (xy)	Orthogonal diagonalization to remove all cross-terms simultaneously
Eigenvalues tell the shape of a curve	Eigenvalues determine stability of systems, natural frequencies of vibration, and principal components of data
Positive/negative definite classification	Sylvester's Law of Inertia and the signature of a quadratic form

Intro vs. Advanced Quadratic Form Theory

In a future lesson, you'll see how the Spectral Theorem guarantees that every real symmetric matrix can be diagonalized using an orthogonal matrix. You'll also explore how eigenvalues help engineers predict whether a bridge will resonate dangerously in the wind, or how data scientists use PCA to compress millions of data points into just a few meaningful principal directions.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

In your own words, explain what a "principal direction" of a matrix means. Why is it special compared to other directions?

Show answer

A principal direction (eigenvector) is a direction where the matrix acts like simple multiplication — it only stretches or compresses the vector without rotating it. If you multiply the matrix A by an eigenvector v, you get λv, which points in the same (or exactly opposite) direction. Most vectors get rotated when multiplied by a matrix, so eigenvectors are special because they resist that rotation. The stretch factor λ (the eigenvalue) tells you how much the vector grows or shrinks.

Question 2: BASIC CALCULATION

Given the quadratic form Q(x, y) = 3x² + 2y², write the associated symmetric matrix A. Then find its eigenvalues. (Hint: there is no xy term.)

Show answer

Since Q = 3x² + 0·xy + 2y², we have a = 3, b = 0, and c = 2. The matrix is A = [[3, 0], [0, 2]]. This matrix is already diagonal! The eigenvalues are just the diagonal entries: λ₁ = 3 and λ₂ = 2. Both are positive, so Q is positive definite and the curve Q = constant is an ellipse. The principal directions are simply the x-axis and y-axis, which makes sense because there's no cross-term to create a tilt.

Question 3: INTERMEDIATE

Find the eigenvalues and eigenvectors of the matrix A = [[2, 1], [1, 2]]. Then classify the quadratic form Q(x, y) = 2x² + 2xy + 2y².

Show answer

Set up det(A − λI) = 0: (2 − λ)² − 1 = 0, so λ² − 4λ + 3 = 0, giving (λ − 1)(λ − 3) = 0. The eigenvalues are λ₁ = 1 and λ₂ = 3. For λ₁ = 1: (A − I)v = 0 gives [[1, 1], [1, 1]]v = 0, so v₁ + v₂ = 0 and v₁ = [1, −1]. For λ₂ = 3: (A − 3I)v = 0 gives [[−1, 1], [1, −1]]v = 0, so v₁ = v₂ and v₂ = [1, 1]. Both eigenvalues are positive, so Q is positive definite. The diagonalized form is Q = u² + 3v², an ellipse.

Question 4: APPLIED

A stress analyst models the internal forces in a metal plate using the quadratic form Q(x, y) = 4x² − 6xy + 4y². Find the eigenvalues of the associated matrix and determine whether the stress produces an elliptical or hyperbolic pattern. What are the principal stress directions?

Show answer

The matrix is A = [[4, −3], [−3, 4]] (since the coefficient of xy is −6, we use b = −3). The characteristic equation is (4 − λ)² − 9 = 0, expanding to λ² − 8λ + 7 = 0, which factors as (λ − 1)(λ − 7) = 0. So λ₁ = 1 and λ₂ = 7. Both are positive, so the quadratic form is positive definite — the stress pattern is elliptical, not hyperbolic. For λ₁ = 1: [[3, −3], [−3, 3]]v = 0 gives v₁ = [1, 1]. For λ₂ = 7: [[−3, −3], [−3, −3]]v = 0 gives v₂ = [1, −1]. The principal stress directions are along [1, 1] (the 45° line) and [1, −1] (the 135° line).

Question 5: CRITICAL THINKING

Suppose a 2×2 symmetric matrix A has eigenvalues λ₁ = k and λ₂ = k (both equal). What shape does the quadratic form Q = xᵀAx describe when Q = constant? Why does the concept of 'principal direction' become less meaningful in this case? Explain your reasoning.

Show answer

When both eigenvalues are equal to k, the diagonalized form becomes Q = ku² + kv² = k(u² + v²). If k > 0, this is a circle (not an ellipse with two different axes). If k < 0, it's still a circle but with Q < 0. A circle has no preferred axis — it looks the same no matter how you rotate it. This means every direction is equally valid as a principal direction. Mathematically, Av = kv is satisfied by every nonzero vector v, since A = kI (k times the identity matrix). The concept of 'principal direction' becomes less meaningful because there's no special direction to discover. This is analogous to how a perfectly round ball has no 'long axis' or 'short axis' — every diameter is the same length.

SUMMARY

Lesson Summary

Summary

A quadratic form is an expression like ax² + 2bxy + cy² that can be written as xᵀAx using a symmetric matrix A. The principal directions are the eigenvectors of A — the special directions along which A acts as a simple stretch. The corresponding eigenvalues tell you the stretch factor in each principal direction.

By switching to the principal directions as new coordinate axes (a process called diagonalization), you eliminate cross-terms and rewrite the quadratic form as λ₁u² + λ₂v². The signs of the eigenvalues instantly classify the curve: both positive means an ellipse, opposite signs mean a hyperbola, and a zero eigenvalue signals a degenerate case. This powerful technique extends to higher dimensions and is the foundation for applications in physics, engineering, and data science.