01 / 20

Projections onto
Lines & Subspaces

Learn how to project vectors onto lines and subspaces: the geometric idea, the formula, and the projection matrix.

Linear Algebra

What Is a Projection?

Imagine shining a flashlight straight down onto a line. The shadow your vector casts on that line is its projection. Formally, the projection of \(\mathbf{b}\) onto a line through \(\mathbf{a}\) is the closest point on that line to \(\mathbf{b}\).

a b proj error O

Why Do Projections Matter?

Least Squares
When \(A\mathbf{x}=\mathbf{b}\) has no solution, the best approximate solution comes from projecting \(\mathbf{b}\) onto the column space of \(A\).
Orthogonal Decomposition
Any vector can be split into a component along a subspace plus a component perpendicular to it.
Signal Processing
Fourier analysis and data compression rely on projecting data onto basis vectors to keep only the important parts.

The Key Idea: Orthogonality

The projection \(\text{proj}_{\mathbf{a}}\,\mathbf{b}\) is the point on the line through \(\mathbf{a}\) such that the error vector is orthogonal to \(\mathbf{a}\). If \(\hat{\mathbf{p}} = \hat{x}\,\mathbf{a}\), then:

This single equation captures the whole idea: set the error perpendicular to the direction of projection, then solve for the scalar \(\hat{x}\).

Projection onto a Line

a b proj_a b e = b − p O

The Projection Matrix

We can rewrite the formula as a matrix times \(\mathbf{b}\):

\(P^2 = P\)
Projecting twice gives the same result: idempotent.
\(P^T = P\)
The projection matrix is symmetric.
rank \(= 1\)
For a line, \(P\) is a rank-1 matrix (outer product).

Projection onto a Subspace

When the target is a subspace spanned by the columns of \(A\) (not just one vector), the same orthogonality idea applies. The error \(\mathbf{b} - A\hat{\mathbf{x}}\) must be perpendicular to every column of \(A\):

This generalises the line formula. When \(A\) is a single column vector \(\mathbf{a}\), it reduces back to \(\frac{\mathbf{a}\mathbf{a}^T}{\mathbf{a}^T\mathbf{a}}\).

Line vs Subspace Projection

Onto a Line Onto a Subspace
Target Line through \(\mathbf{a}\) Col space of \(A\)
Scalar \(\hat{x} = \frac{\mathbf{a}\cdot\mathbf{b}}{\mathbf{a}\cdot\mathbf{a}}\) \(\hat{\mathbf{x}} = (A^TA)^{-1}A^T\mathbf{b}\)
Proj Matrix \(\frac{\mathbf{a}\mathbf{a}^T}{\mathbf{a}^T\mathbf{a}}\) \(A(A^TA)^{-1}A^T\)
Rank of \(P\) 1 \(\text{rank}(A)\)
Properties \(P^2=P,\; P^T=P\) \(P^2=P,\; P^T=P\)

How to Compute a Projection

1
Identify the target. Is it a line (one vector \(\mathbf{a}\)) or a subspace (matrix \(A\))?
2
Compute the dot products. For a line: \(\mathbf{a}\cdot\mathbf{b}\) and \(\mathbf{a}\cdot\mathbf{a}\). For a subspace: \(A^T A\) and \(A^T\mathbf{b}\).
3
Solve for \(\hat{\mathbf{x}}\). Divide (line) or solve the normal equations (subspace).
4
Find the projection. \(\mathbf{p} = \hat{x}\,\mathbf{a}\) or \(\mathbf{p} = A\hat{\mathbf{x}}\).
5
Verify orthogonality. Check that \(\mathbf{a}\cdot(\mathbf{b}-\mathbf{p})=0\) or \(A^T(\mathbf{b}-\mathbf{p})=\mathbf{0}\).

Example 1: Projection onto a Line

Project \(\mathbf{b} = \begin{pmatrix}1\\2\\2\end{pmatrix}\) onto the line through \(\mathbf{a} = \begin{pmatrix}1\\0\\1\end{pmatrix}\).

1
Dot products: \(\mathbf{a}\cdot\mathbf{b} = 1(1)+0(2)+1(2) = \) 3,   \(\mathbf{a}\cdot\mathbf{a} = 1+0+1 = \) 2
2
Scalar: \(\hat{x} = \frac{3}{2}\)
3
Projection: \(\mathbf{p} = \frac{3}{2}\begin{pmatrix}1\\0\\1\end{pmatrix} = \) \(\begin{pmatrix}3/2\\0\\3/2\end{pmatrix}\)

Example 2: Projection Matrix

Find the projection matrix \(P\) that projects onto the line through \(\mathbf{a} = \begin{pmatrix}2\\1\end{pmatrix}\).

1
Outer product: \(\mathbf{a}\mathbf{a}^T = \begin{pmatrix}2\\1\end{pmatrix}\begin{pmatrix}2&1\end{pmatrix} = \begin{pmatrix}4&2\\2&1\end{pmatrix}\)
2
Denominator: \(\mathbf{a}^T\mathbf{a} = 4+1 = 5\)
3
Verify: \(P^2 = \frac{1}{25}\begin{pmatrix}4&2\\2&1\end{pmatrix}^2 = \frac{1}{25}\begin{pmatrix}20&10\\10&5\end{pmatrix} = \frac{1}{5}\begin{pmatrix}4&2\\2&1\end{pmatrix} = P\) ✓

Example 3: Onto a Plane

Project \(\mathbf{b}=\begin{pmatrix}1\\1\\1\end{pmatrix}\) onto the column space of \(A=\begin{pmatrix}1&0\\0&1\\0&0\end{pmatrix}\).

1
Normal equation: \(A^TA = \begin{pmatrix}1&0\\0&1\end{pmatrix}\),   \(A^T\mathbf{b} = \begin{pmatrix}1\\1\end{pmatrix}\)
2
Solve: \(\hat{\mathbf{x}} = \begin{pmatrix}1\\1\end{pmatrix}\)
3
Projection: \(\mathbf{p} = A\hat{\mathbf{x}} = \) \(\begin{pmatrix}1\\1\\0\end{pmatrix}\)

Visualising Subspace Projection

Col(A) O b p e

Properties of \(P\)

\(P^2 = P\)
Idempotent: once a vector is on the subspace, projecting again changes nothing.
\(P^T = P\)
Symmetric: the matrix equals its own transpose.
Eigenvalues
Only \(0\) and \(1\). Vectors in the subspace get eigenvalue 1; vectors perpendicular get 0.
\(I - P\)
Projects onto the orthogonal complement: it extracts the error component.

Common Mistakes

✕ Dividing vectors
Writing \(\frac{\mathbf{b}}{\mathbf{a}}\). You cannot divide vectors. Use the dot product formula instead.
✕ Wrong order in \(P\)
Writing \(\mathbf{a}^T\mathbf{a}\) in the numerator. The outer product \(\mathbf{a}\mathbf{a}^T\) gives a matrix; \(\mathbf{a}^T\mathbf{a}\) is a scalar.
✕ Forgetting \((A^TA)^{-1}\)
Writing \(P = AA^T\). You must include \((A^TA)^{-1}\) between them unless the columns of \(A\) are orthonormal.

Summary Cheat Sheet

Quantity Formula Notes
Proj onto line \(\frac{\mathbf{a}\cdot\mathbf{b}}{\mathbf{a}\cdot\mathbf{a}}\,\mathbf{a}\) Scalar times vector
\(P\) (line) \(\frac{\mathbf{a}\mathbf{a}^T}{\mathbf{a}^T\mathbf{a}}\) Rank 1, symmetric
Normal eq. \(A^TA\hat{\mathbf{x}} = A^T\mathbf{b}\) Key equation
\(P\) (subspace) \(A(A^TA)^{-1}A^T\) Rank = rank(\(A\))
Error \(\mathbf{e}=\mathbf{b}-P\mathbf{b}\) \(\mathbf{e}\perp\) col(\(A\))

Question 1 of 5

What is the projection of \(\mathbf{b}=\begin{pmatrix}3\\4\end{pmatrix}\) onto the line through \(\mathbf{a}=\begin{pmatrix}1\\0\end{pmatrix}\)?

Question 2 of 5

If \(P\) is a projection matrix, which property must it satisfy?

Question 3 of 5

The error vector \(\mathbf{e} = \mathbf{b} - \text{proj}_{\mathbf{a}}\,\mathbf{b}\) is always:

Question 4 of 5

What is the projection matrix \(P\) that projects onto the line through \(\mathbf{a}=\begin{pmatrix}1\\1\end{pmatrix}\)?

Question 5 of 5

For the subspace formula \(P = A(A^TA)^{-1}A^T\), what does \((A^TA)^{-1}\) require?