01 / 21

Gram-Schmidt
Orthonormalization

Transform any set of linearly independent vectors into an orthonormal basis using projection and normalization.

Linear Algebra

What Is an Orthonormal Basis?

A set of vectors \(\{e_1, e_2, \ldots, e_n\}\) forms an orthonormal basis if every pair is orthogonal (perpendicular) and every vector has unit length:

Orthogonal
\(\langle e_i, e_j \rangle = 0\) whenever \(i \neq j\)
Normal
\(\|e_i\| = 1\) for every basis vector

Why Orthonormal Bases Matter

Working in an orthonormal basis simplifies nearly every computation in linear algebra. Coordinates, projections, and decompositions all become trivial.

Easy Coordinates
The coordinate of \(v\) along \(e_i\) is simply \(\langle v, e_i \rangle\): no system to solve.
Preserved Lengths
Orthonormal transforms preserve inner products, so \(\|Qv\| = \|v\|\).
QR Factorization
Gram-Schmidt produces the \(QR\) decomposition used in eigenvalue algorithms.

The Projection Idea

The heart of Gram-Schmidt is vector projection. To remove the component of \(v\) along a direction \(u\), subtract its projection:

\(u\)
\(v\)
\(v - \operatorname{proj}\)
\(\operatorname{proj}_u(v)\)

The dashed line is the residual: the part of \(v\) perpendicular to \(u\). Gram-Schmidt collects these residuals one at a time.

The Gram-Schmidt Formula

Given linearly independent vectors \(\{v_1, v_2, \ldots, v_n\}\), produce orthogonal vectors \(\{u_1, u_2, \ldots, u_n\}\) and then normalize:

1
Set \(u_1 = v_1\)
2
For each \(k = 2, \ldots, n\):
\(\displaystyle u_k = v_k - \sum_{j=1}^{k-1} \frac{\langle v_k, u_j \rangle}{\langle u_j, u_j \rangle}\, u_j\)
3
Normalize: \(\displaystyle e_k = \frac{u_k}{\|u_k\|}\) for each \(k\)

Visualizing the Process: Two Vectors

Before
\(v_1\)
\(v_2\)
\(\theta\)
 After Gram-Schmidt
\(e_1\)
\(e_2\)
\(90^\circ\)

Orthogonal vs Orthonormal

Orthogonal Set Orthonormal Set
Inner product \(\langle u_i, u_j \rangle = 0\) for \(i \neq j\) \(\langle e_i, e_j \rangle = 0\) for \(i \neq j\)
Vector length Any nonzero length \(\|e_i\| = 1\) exactly
Projection formula \(\displaystyle \frac{\langle v,u\rangle}{\langle u,u\rangle}u\) \(\langle v,e\rangle\, e\) (simpler!)
Matrix form \(Q^TQ\) is diagonal \(Q^TQ = I\) (identity)

How to Apply Gram-Schmidt

1
Label your input vectors \(v_1, v_2, \ldots, v_n\).
2
Set \(u_1 = v_1\). This is your first orthogonal vector.
3
Compute projections: for each new \(v_k\), compute \(\operatorname{proj}_{u_j}(v_k)\) for all \(j < k\).
4
Subtract all projections: \(u_k = v_k - \sum \operatorname{proj}_{u_j}(v_k)\).
5
Normalize: \(e_k = u_k / \|u_k\|\). Repeat steps 3-5 for each remaining vector.
6
Verify: check that \(\langle e_i, e_j \rangle = 0\) for \(i \neq j\) and \(\|e_i\| = 1\).

Example 1: Two Vectors in \(\mathbb{R}^2\)

Orthonormalize \(v_1 = \begin{pmatrix}3\\1\end{pmatrix}\) and \(v_2 = \begin{pmatrix}2\\2\end{pmatrix}\).

1
Set \(u_1 = v_1 = \begin{pmatrix}3\\1\end{pmatrix}\).
2
Compute projection:
\(\displaystyle \operatorname{proj}_{u_1}(v_2) = \frac{\langle v_2, u_1 \rangle}{\langle u_1, u_1 \rangle}\, u_1 = \frac{2\cdot3 + 2\cdot1}{9+1}\begin{pmatrix}3\\1\end{pmatrix} = \frac{8}{10}\begin{pmatrix}3\\1\end{pmatrix} = \begin{pmatrix}12/5\\4/5\end{pmatrix}\)
3
Subtract:
\(\displaystyle u_2 = v_2 - \operatorname{proj}_{u_1}(v_2) = \begin{pmatrix}2\\2\end{pmatrix} - \begin{pmatrix}12/5\\4/5\end{pmatrix} = \begin{pmatrix}-2/5\\6/5\end{pmatrix}\)

Example 1: Normalize

4
Normalize \(u_1\):
\(\|u_1\| = \sqrt{9+1} = \sqrt{10}\), so \(\displaystyle e_1 = \frac{1}{\sqrt{10}}\begin{pmatrix}3\\1\end{pmatrix}\)
5
Normalize \(u_2\):
\(\|u_2\| = \sqrt{4/25 + 36/25} = \sqrt{40/25} = \frac{2\sqrt{10}}{5}\), so \(\displaystyle e_2 = \frac{5}{2\sqrt{10}}\begin{pmatrix}-2/5\\6/5\end{pmatrix} = \frac{1}{\sqrt{10}}\begin{pmatrix}-1\\3\end{pmatrix}\)

Example 2: Three Vectors in \(\mathbb{R}^3\)

Orthonormalize: \(v_1 = \begin{pmatrix}1\\1\\0\end{pmatrix}\), \(v_2 = \begin{pmatrix}1\\0\\1\end{pmatrix}\), \(v_3 = \begin{pmatrix}0\\1\\1\end{pmatrix}\).

1
\(u_1 = v_1 = \begin{pmatrix}1\\1\\0\end{pmatrix}\), with \(\|u_1\|^2 = 2\).
2
For \(u_2\):
\(\langle v_2, u_1 \rangle = 1\), so \(\operatorname{proj}_{u_1}(v_2) = \frac{1}{2}\begin{pmatrix}1\\1\\0\end{pmatrix}\).
\(u_2 = \begin{pmatrix}1\\0\\1\end{pmatrix} - \begin{pmatrix}1/2\\1/2\\0\end{pmatrix} = \begin{pmatrix}1/2\\-1/2\\1\end{pmatrix}\), with \(\|u_2\|^2 = \frac{3}{2}\).

Example 2: Finding \(u_3\) and Normalizing

3
For \(u_3\):
\(\langle v_3, u_1 \rangle = 1\), \(\operatorname{proj}_{u_1}(v_3) = \frac{1}{2}\begin{pmatrix}1\\1\\0\end{pmatrix}\).
\(\langle v_3, u_2 \rangle = -\frac{1}{2}+1 = \frac{1}{2}\), \(\operatorname{proj}_{u_2}(v_3) = \frac{1/2}{3/2}\begin{pmatrix}1/2\\-1/2\\1\end{pmatrix} = \begin{pmatrix}1/6\\-1/6\\1/3\end{pmatrix}\).
\(u_3 = \begin{pmatrix}0\\1\\1\end{pmatrix} - \begin{pmatrix}1/2\\1/2\\0\end{pmatrix} - \begin{pmatrix}1/6\\-1/6\\1/3\end{pmatrix} = \begin{pmatrix}-2/3\\2/3\\2/3\end{pmatrix}\).
4
Normalize all three:
\(e_1 = \frac{1}{\sqrt{2}}\begin{pmatrix}1\\1\\0\end{pmatrix}\), \(e_2 = \sqrt{\frac{2}{3}}\begin{pmatrix}1/2\\-1/2\\1\end{pmatrix}\), \(e_3 = \frac{1}{2/\sqrt{3}}\begin{pmatrix}-2/3\\2/3\\2/3\end{pmatrix} = \frac{1}{\sqrt{3}}\begin{pmatrix}-1\\1\\1\end{pmatrix}\).

Connection to \(QR\) Decomposition

If we arrange the original vectors as columns of a matrix \(A = [v_1 \mid v_2 \mid \cdots \mid v_n]\), the Gram-Schmidt process yields:

\(Q\)
Columns are the orthonormal vectors from Gram-Schmidt. \(Q^TQ = I\).
\(R\)
Entries are the inner products and norms: \(r_{jk} = \langle v_k, e_j \rangle\), with \(r_{kk} = \|u_k\|\).

The \(QR\) factorization is essential in least-squares problems, eigenvalue algorithms, and numerical stability.

Geometric View in \(\mathbb{R}^3\)

\(e_1\)
\(e_2\)
\(e_3\)
\(v_2\)
\(O\)
 All three axes are mutually perpendicular

Common Mistakes

✕ Forgetting to Subtract All Projections
When computing \(u_k\), you must subtract projections onto every previous \(u_j\) (\(j = 1, \ldots, k-1\)), not just the last one.
✕ Normalizing Too Early
Do not normalize \(u_k\) before you have finished using it in later projection computations, or switch to the normalized projection formula.
✕ Wrong Denominator
In \(\operatorname{proj}_{u}(v)\), the denominator is \(\langle u, u \rangle\), not \(\langle v, v \rangle\). Swapping gives the wrong projection.

Summary Table

Step Operation Result
1 \(u_1 = v_1\) First orthogonal vector
2 \(u_k = v_k - \sum \operatorname{proj}_{u_j}(v_k)\) Remove all previous directions
3 \(e_k = u_k / \|u_k\|\) Scale to unit length
4 Verify \(\langle e_i, e_j \rangle = \delta_{ij}\) Confirm orthonormality
5 Build \(Q = [e_1 \mid \cdots \mid e_n]\) \(A = QR\) decomposition

Question 1 of 5

What does the Gram-Schmidt process produce from a set of linearly independent vectors?

Question 2 of 5

Given \(u = \begin{pmatrix}2\\1\end{pmatrix}\) and \(v = \begin{pmatrix}1\\3\end{pmatrix}\), what is \(\operatorname{proj}_{u}(v)\)?

Question 3 of 5

If \(u_1\) and \(u_2\) are the orthogonal vectors produced by Gram-Schmidt, which property must hold?

Question 4 of 5

In the \(QR\) decomposition \(A = QR\), what is the structure of the matrix \(R\)?

Question 5 of 5

When computing \(u_3\) in Gram-Schmidt, how many projection terms must you subtract from \(v_3\)?