01 / 21

Principal Directions
& Quadratic Forms

Discover how symmetric matrices determine the shape and orientation of quadratic forms, and learn to find their principal axes through eigenvalues and eigenvectors.

Linear Algebra

What Is a Quadratic Form?

A quadratic form is a scalar-valued function of a vector \(\mathbf{x} \in \mathbb{R}^n\) defined by a symmetric matrix \(A\):

For \(\mathbf{x} = \begin{pmatrix} x_1 \\ x_2 \end{pmatrix}\) and \(A = \begin{pmatrix} a & b \\ b & c \end{pmatrix}\), this expands to:

Visualizing Level Curves

Setting \(Q(\mathbf{x}) = c\) for constant \(c\) produces level curves: ellipses, hyperbolas, or lines depending on \(A\). The shape reveals how the matrix "stretches" space.

\(x_1\) \(x_2\) \(\mathbf{v}_1\) \(\mathbf{v}_2\) Level curves of \(Q = c\)

Why Principal Directions Matter

Simplification
In the eigenvector basis, all cross terms vanish: \(Q = \lambda_1 y_1^2 + \lambda_2 y_2^2\).
Geometry
They reveal the natural axes of the ellipse, hyperbola, or paraboloid described by \(Q\).
Applications
Used in PCA, stress tensors, optimization, and classifying critical points via the Hessian.

The Spectral Theorem

Every real symmetric matrix \(A\) can be orthogonally diagonalized:

1
\(P = [\mathbf{v}_1 \mid \mathbf{v}_2 \mid \cdots \mid \mathbf{v}_n]\) is an orthogonal matrix whose columns are the eigenvectors (principal directions).
2
\(\Lambda = \operatorname{diag}(\lambda_1, \lambda_2, \ldots, \lambda_n)\) is the diagonal matrix of eigenvalues.
3
\(P^T P = I\): the eigenvectors form an orthonormal basis.

Diagonalizing the Quadratic Form

Substituting \(\mathbf{x} = P\mathbf{y}\) (a change to the eigenvector basis) into \(Q\):

So in the new coordinates \(\mathbf{y}\), the quadratic form becomes:

No cross terms remain. The eigenvalues tell you the curvature along each principal direction.

Definite vs Indefinite Forms

Positive Definite
All \(\lambda_i > 0\). Level curves are ellipses. \(Q > 0\) for all \(\mathbf{x} \neq \mathbf{0}\).
Positive Semi-Definite
All \(\lambda_i \ge 0\). \(Q \ge 0\) everywhere, but equals 0 on a subspace.
Indefinite
Mixed signs: some \(\lambda_i > 0\), some \(\lambda_j < 0\). Level curves are hyperbolas.
Negative Definite
All \(\lambda_i < 0\). \(Q < 0\) for all \(\mathbf{x} \neq \mathbf{0}\). Inverted ellipses.

How to Find Principal Directions

1
Write down \(A\): extract the symmetric matrix from the quadratic form. Remember: the coefficient of \(x_i x_j\) (for \(i \neq j\)) is split equally as \(a_{ij} = a_{ji}\).
2
Solve \(\det(A - \lambda I) = 0\): find all eigenvalues \(\lambda_1, \lambda_2, \ldots\)
3
Find eigenvectors: for each \(\lambda_i\), solve \((A - \lambda_i I)\mathbf{v} = \mathbf{0}\).
4
Normalize & form \(P\): make each eigenvector unit length. The columns of \(P\) are the principal directions.
5
Write the diagonal form: \(Q = \lambda_1 y_1^2 + \lambda_2 y_2^2\) in the rotated coordinates \(\mathbf{y} = P^T \mathbf{x}\).

Example 1: Diagonalize a Quadratic Form

Given the quadratic form:

Step 1: The associated symmetric matrix is:

Note: the coefficient of \(x_1 x_2\) is 4, so \(a_{12} = a_{21} = \frac{4}{2} = 2\).

Example 1: Finding Eigenvalues

Step 2: Solve \(\det(A - \lambda I) = 0\):

1
Expand: \(\lambda^2 - 10\lambda + 21 = 0\)
2
Factor: \((\lambda - 7)(\lambda - 3) = 0\)
3
Eigenvalues: \(\lambda_1 = 7\), \(\lambda_2 = 3\)

Example 1: Principal Directions

Step 3: For \(\lambda_1=7\): solve \((A-7I)\mathbf{v}=\mathbf{0}\), giving \(\mathbf{v}_1 = \frac{1}{\sqrt{2}}\begin{pmatrix}1\\1\end{pmatrix}\). For \(\lambda_2=3\): \(\mathbf{v}_2 = \frac{1}{\sqrt{2}}\begin{pmatrix}1\\-1\end{pmatrix}\).

\(x_1\) \(x_2\) \(\mathbf{v}_1\) \(\mathbf{v}_2\) \(\lambda_1=7\): shorter axis \(\lambda_2=3\): longer axis

Example 2: An Indefinite Form

Consider \(Q(x_1, x_2) = x_1^2 - 6x_1 x_2 + x_2^2\). The matrix is:

Characteristic equation: \((1-\lambda)^2 - 9 = 0 \implies \lambda^2 - 2\lambda - 8 = 0\).

1
Factor: \((\lambda - 4)(\lambda + 2) = 0\)
2
Eigenvalues: \(\lambda_1 = 4\) and \(\lambda_2 = -2\)
3
Indefinite (mixed signs): level curves are hyperbolas.

Example 2: Hyperbolic Level Curves

\(x_1\) \(x_2\) \(\mathbf{v}_1\) (\(\lambda=4\)) \(\mathbf{v}_2\) (\(\lambda=-2\))

Example 3: Three Dimensions

For \(Q = 2x_1^2 + 2x_2^2 + 2x_3^2 + 2x_1x_2\), the symmetric matrix is:

1
By inspection or calculation: \(\det(A - \lambda I) = (2-\lambda)[(2-\lambda)^2 - 1] = 0\)
2
Eigenvalues: \(\lambda_1 = 3\), \(\lambda_2 = 2\), \(\lambda_3 = 1\)
3
All positive: positive definite. Level surface \(Q = c\) is an ellipsoid.

Common Mistakes

✕ Wrong Off-Diagonal Entry
Putting the full coefficient of \(x_1 x_2\) (e.g., 4) as \(a_{12}\) instead of splitting it: \(a_{12} = a_{21} = 2\).
✕ Non-Symmetric Matrix
Writing \(A\) with \(a_{12} \neq a_{21}\). The spectral theorem requires \(A = A^T\).
✕ Forgetting to Normalize
Using eigenvectors that aren't unit length. The change of basis \(P\) must be orthogonal: \(P^T P = I\).

Classification Summary

Type Eigenvalue Signs Level Curves (2D) \(Q(\mathbf{x})\) for \(\mathbf{x}\neq\mathbf{0}\)
Positive Definite All \(\lambda_i > 0\) Ellipses \(Q > 0\)
Pos. Semi-Definite All \(\lambda_i \ge 0\) Parallel lines \(Q \ge 0\)
Indefinite Mixed signs Hyperbolas Both signs possible
Negative Definite All \(\lambda_i < 0\) Ellipses (inverted) \(Q < 0\)
Neg. Semi-Definite All \(\lambda_i \le 0\) Parallel lines \(Q \le 0\)

Question 1 of 5

What is the symmetric matrix \(A\) associated with the quadratic form \(Q = 3x_1^2 + 8x_1 x_2 + x_2^2\)?

Question 2 of 5

If a \(2\times 2\) symmetric matrix has eigenvalues \(\lambda_1 = 5\) and \(\lambda_2 = -1\), what type is the associated quadratic form?

Question 3 of 5

After diagonalizing \(Q(\mathbf{x}) = \mathbf{x}^T A \mathbf{x}\) via the substitution \(\mathbf{x} = P\mathbf{y}\), the quadratic form becomes:

Question 4 of 5

For \(A = \begin{pmatrix} 4 & 2 \\ 2 & 1 \end{pmatrix}\), the eigenvalues are \(\lambda_1 = 5\) and \(\lambda_2 = 0\). What is the classification of \(\mathbf{x}^T A \mathbf{x}\)?

Question 5 of 5

In the diagonal form \(Q = 7y_1^2 + 3y_2^2\), which principal direction has greater curvature (shorter semi-axis on the level ellipse)?