01 / 20

Orthogonality &
Complements

Explore when vectors are perpendicular, how to decompose spaces into orthogonal pieces, and why the dot product \(\mathbf{u} \cdot \mathbf{v} = 0\) unlocks powerful tools in linear algebra.

Linear Algebra

What Is Orthogonality?

Two vectors \(\mathbf{u}\) and \(\mathbf{v}\) in \(\mathbb{R}^n\) are orthogonal if their dot product equals zero:

Geometrically, orthogonal vectors meet at a right angle (\(90^\circ\)). The zero vector \(\mathbf{0}\) is orthogonal to every vector.

Orthogonal Vectors in \(\mathbb{R}^2\)

x y \(\mathbf{u} = \begin{pmatrix}3\\1\end{pmatrix}\) \(\mathbf{v} = \begin{pmatrix}-1\\3\end{pmatrix}\) O

Why Does Orthogonality Matter?

Simplifies Projections
Projecting onto orthogonal bases requires only dot products: no systems of equations to solve.
Decomposes Spaces
Every vector in \(\mathbb{R}^n\) splits uniquely into a part inside a subspace and a part in its orthogonal complement.
Powers Applications
Least-squares regression, QR factorization, Fourier analysis, and PCA all rely on orthogonality.

The Orthogonal Complement \(W^\perp\)

\(W^\perp\) collects every vector that is perpendicular to every vector in \(W\). It is itself a subspace of \(\mathbb{R}^n\).

Visualizing \(W\) and \(W^\perp\) in \(\mathbb{R}^3\)

\(W\) (plane) \(W^\perp\) (line) \(\dim = 2\) \(\dim = 1\) \(2 + 1 = 3\) ✓

If \(W\) is a plane through the origin in \(\mathbb{R}^3\) (dimension 2), then \(W^\perp\) is the line perpendicular to that plane (dimension 1). Together they account for all of \(\mathbb{R}^3\).

Orthogonal Decomposition Theorem

\(W\) \(\hat{\mathbf{y}}\) (proj onto \(W\)) \(\mathbf{z} \in W^\perp\) \(\mathbf{y}\)

The component \(\hat{\mathbf{y}}\) is the orthogonal projection of \(\mathbf{y}\) onto \(W\), and \(\mathbf{z} = \mathbf{y} - \hat{\mathbf{y}}\) is the residual in \(W^\perp\).

Computing the Projection

If \(W = \text{Span}\{\mathbf{u}_1, \ldots, \mathbf{u}_p\}\) and the \(\mathbf{u}_i\) are orthogonal to each other, then:

Orthogonal vs Orthonormal

Property Orthogonal Set Orthonormal Set
Dot product of distinct vectors \(\mathbf{u}_i \cdot \mathbf{u}_j = 0\) \(\mathbf{u}_i \cdot \mathbf{u}_j = 0\)
Length of each vector Any nonzero length \(\|\mathbf{u}_i\| = 1\)
Projection formula \(\dfrac{\mathbf{y} \cdot \mathbf{u}_i}{\mathbf{u}_i \cdot \mathbf{u}_i}\,\mathbf{u}_i\) \((\mathbf{y} \cdot \mathbf{u}_i)\,\mathbf{u}_i\)
How to obtain Gram-Schmidt (raw output) Normalize each vector

The Fundamental Subspace Connections

For any \(m \times n\) matrix \(A\), the four fundamental subspaces pair up as orthogonal complements:

In \(\mathbb{R}^n\)
\(\text{Row}(A)^\perp = \text{Nul}(A)\)
Row space and null space are complements.
In \(\mathbb{R}^m\)
\(\text{Col}(A)^\perp = \text{Nul}(A^T)\)
Column space and left null space are complements.

How to Find \(W^\perp\)

1
Express \(W\) as a row space. If \(W = \text{Span}\{\mathbf{w}_1, \ldots, \mathbf{w}_k\}\), form matrix \(A\) with those vectors as rows.
2
Row reduce \(A\). Bring \(A\) to reduced row echelon form (RREF).
3
Solve \(A\mathbf{x} = \mathbf{0}\). Find the null space: parametrize the free variables.
4
Report the null space basis. Those vectors span \(W^\perp = \text{Nul}(A)\).

Example 1: Orthogonality Check

Are \(\mathbf{u} = \begin{bmatrix} 2 \\ -3 \\ 1 \end{bmatrix}\) and \(\mathbf{v} = \begin{bmatrix} 1 \\ 2 \\ 4 \end{bmatrix}\) orthogonal?

1
Compute the dot product: \(\mathbf{u} \cdot \mathbf{v} = (2)(1) + (-3)(2) + (1)(4)\)
2
Simplify: \(= 2 - 6 + 4 = 0\)
3
Conclude: Since \(\mathbf{u} \cdot \mathbf{v} = 0\), the vectors are orthogonal. ✓

Example 2: Finding \(W^\perp\)

Let \(W = \text{Span}\left\{\begin{bmatrix}1\\2\\-1\end{bmatrix},\;\begin{bmatrix}0\\1\\3\end{bmatrix}\right\}\) in \(\mathbb{R}^3\). Find \(W^\perp\).

1
Form matrix \(A\): \(A = \begin{bmatrix}1 & 2 & -1 \\ 0 & 1 & 3\end{bmatrix}\)
2
RREF: \(R_1 \leftarrow R_1 - 2R_2\) gives \(\begin{bmatrix}1 & 0 & -7 \\ 0 & 1 & 3\end{bmatrix}\)
3
Solve \(A\mathbf{x} = \mathbf{0}\): Free variable \(x_3 = t\). So \(x_1 = 7t\), \(x_2 = -3t\).
4
Result: \(W^\perp = \text{Span}\left\{\begin{bmatrix}7\\-3\\1\end{bmatrix}\right\}\). Dimension: \(3 - 2 = 1\) ✓

Example 3: Projecting onto \(W\)

Let \(\mathbf{u}_1 = \begin{bmatrix}1\\1\\0\end{bmatrix}\), \(\mathbf{u}_2 = \begin{bmatrix}-1\\1\\0\end{bmatrix}\) (orthogonal), \(W = \text{Span}\{\mathbf{u}_1, \mathbf{u}_2\}\). Find \(\text{proj}_W \mathbf{y}\) for \(\mathbf{y} = \begin{bmatrix}3\\5\\7\end{bmatrix}\).

1
Coefficient 1: \(\dfrac{\mathbf{y} \cdot \mathbf{u}_1}{\mathbf{u}_1 \cdot \mathbf{u}_1} = \dfrac{3+5+0}{1+1+0} = \dfrac{8}{2} = 4\)
2
Coefficient 2: \(\dfrac{\mathbf{y} \cdot \mathbf{u}_2}{\mathbf{u}_2 \cdot \mathbf{u}_2} = \dfrac{-3+5+0}{1+1+0} = \dfrac{2}{2} = 1\)
3
Projection: \(\hat{\mathbf{y}} = 4\begin{bmatrix}1\\1\\0\end{bmatrix} + 1\begin{bmatrix}-1\\1\\0\end{bmatrix} = \begin{bmatrix}3\\5\\0\end{bmatrix}\)

Common Mistakes

✕ Forgetting the denominator
Writing \((\mathbf{y} \cdot \mathbf{u})\,\mathbf{u}\) instead of \(\dfrac{\mathbf{y} \cdot \mathbf{u}}{\mathbf{u} \cdot \mathbf{u}}\,\mathbf{u}\) when the basis is orthogonal but not orthonormal.
✕ Non-orthogonal basis
Applying the projection sum formula to a basis that is not orthogonal. The formula only works when \(\mathbf{u}_i \cdot \mathbf{u}_j = 0\) for \(i \neq j\).
✕ Confusing \(W\) and \(W^\perp\)
Claiming \(W^\perp\) is found by "making vectors perpendicular" within \(W\). Instead, \(W^\perp\) lives in the ambient space and is perpendicular to all of \(W\).

Summary: Key Facts

Concept Statement
Orthogonal vectors \(\mathbf{u} \cdot \mathbf{v} = 0\)
Orthogonal complement \(W^\perp = \{\mathbf{x} : \mathbf{x} \cdot \mathbf{w} = 0,\;\forall \mathbf{w} \in W\}\)
Dimension theorem \(\dim W + \dim W^\perp = n\)
Decomposition \(\mathbf{y} = \text{proj}_W \mathbf{y} + \mathbf{z},\; \mathbf{z} \in W^\perp\)
Row/Null duality \(\text{Row}(A)^\perp = \text{Nul}(A)\)
Col/Left-null duality \(\text{Col}(A)^\perp = \text{Nul}(A^T)\)

Question 1 of 5

Which condition makes \(\mathbf{u}\) and \(\mathbf{v}\) orthogonal?

Question 2 of 5

If \(W\) is a 3-dimensional subspace of \(\mathbb{R}^5\), what is \(\dim(W^\perp)\)?

Question 3 of 5

The null space of a matrix \(A\) is the orthogonal complement of which subspace?

Question 4 of 5

Let \(\mathbf{u} = \begin{bmatrix}1\\0\\1\end{bmatrix}\) and \(W = \text{Span}\{\mathbf{u}\}\). Compute \(\text{proj}_W \begin{bmatrix}4\\3\\2\end{bmatrix}\).

Question 5 of 5

A set of vectors \(\{\mathbf{u}_1, \mathbf{u}_2, \mathbf{u}_3\}\) is orthonormal. Which statement is always true?