01 / 21

Vector Norms
& Distance

Learn how to measure the length of a vector and the distance between points using norms: the fundamental ruler of linear algebra.

Linear Algebra

What Is a Norm?

A norm is a function that assigns a non-negative length (or size) to every vector in a vector space. Think of it as a ruler for vectors: it tells you "how big" a vector is, regardless of its direction.

Visualizing a Vector's Length

1 2 3 1 2 2 3 \(\mathbf{v} = (3, 2)\) \(\|\mathbf{v}\| = \sqrt{13}\) O

Why Do Norms Matter?

Measuring Distance
Norms let us compute exact distances between points in any number of dimensions.
Unit Vectors
Dividing by the norm gives a unit vector: same direction, length exactly 1.
Error & Optimization
Machine learning uses norms to measure prediction error and regularize models.

The Euclidean Norm \((\ell^2)\)

The most common norm is the Euclidean norm (also called the \(\ell^2\) norm). For a vector \(\mathbf{v} = (v_1, v_2, \ldots, v_n)\) in \(\mathbb{R}^n\):

This is the straight-line length of the vector: exactly the Pythagorean theorem generalized to \(n\) dimensions.

Three Norm Axioms

1
Non-negativity: \(\|\mathbf{v}\| \geq 0\), and \(\|\mathbf{v}\| = 0 \iff \mathbf{v} = \mathbf{0}\).
2
Scalar multiplication: \(\|c\,\mathbf{v}\| = |c|\,\|\mathbf{v}\|\) for any scalar \(c\).
3
Triangle inequality: \(\|\mathbf{u} + \mathbf{v}\| \leq \|\mathbf{u}\| + \|\mathbf{v}\|\).

Distance Between Two Points

The distance between points \(\mathbf{p}\) and \(\mathbf{q}\) is simply the norm of their difference vector:

P(1,1) Q(4,5) 3 4 d = 5 1 2 3 4 1 2 3 4 5

Comparing Norms: \(\ell^1\), \(\ell^2\), \(\ell^\infty\)

\(\ell^1\) (Manhattan)
\(\|\mathbf{v}\|_1 = |v_1| + |v_2| + \cdots + |v_n|\)
Sum of absolute values.
\(\ell^2\) (Euclidean)
\(\|\mathbf{v}\|_2 = \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2}\)
Straight-line length.
\(\ell^\infty\) (Max)
\(\|\mathbf{v}\|_\infty = \max(|v_1|, |v_2|, \ldots, |v_n|)\)
Largest component.

Unit Balls in Different Norms

\(\ell^1\) \(\ell^2\) \(\ell^\infty\)

How to Compute Norms & Distance

1
Identify the vector (or compute the difference \(\mathbf{p} - \mathbf{q}\) for distance).
2
Choose your norm: \(\ell^1\), \(\ell^2\), or \(\ell^\infty\).
3
Apply the formula: square (then sum and root for \(\ell^2\)), sum absolutes for \(\ell^1\), or take the max for \(\ell^\infty\).
4
Simplify and verify your answer is non-negative.

Example 1: \(\ell^2\) Norm

Find \(\|\mathbf{v}\|_2\) for \(\mathbf{v} = (3, -4)\).

1
Square each component: \(3^2 = 9\), \((-4)^2 = 16\).
2
Sum the squares: \(9 + 16 = 25\).
3
Take the square root: \(\sqrt{25} = 5\).

Example 2: Distance in \(\mathbb{R}^3\)

Find the Euclidean distance between \(\mathbf{p} = (1, 2, 3)\) and \(\mathbf{q} = (4, 6, 3)\).

1
Difference vector: \(\mathbf{p} - \mathbf{q} = (-3, -4, 0)\).
2
Square each component: \(9 + 16 + 0 = 25\).
3
Square root: \(\sqrt{25} = 5\).

Example 3: Three Norms of One Vector

Compute \(\|\mathbf{v}\|_1\), \(\|\mathbf{v}\|_2\), and \(\|\mathbf{v}\|_\infty\) for \(\mathbf{v} = (1, -2, 2)\).

\(\ell^1\)
\(|1| + |-2| + |2|\)
= 5
\(\ell^2\)
\(\sqrt{1 + 4 + 4}\)
= 3
\(\ell^\infty\)
\(\max(1, 2, 2)\)
= 2

Unit Vectors & Normalization

A unit vector has norm exactly 1. To normalize any nonzero vector \(\mathbf{v}\), divide by its norm:

Example: For \(\mathbf{v} = (3, 4)\), we have \(\|\mathbf{v}\|_2 = 5\), so \(\hat{\mathbf{v}} = \left(\frac{3}{5}, \frac{4}{5}\right)\). Verify: \(\sqrt{\left(\frac{3}{5}\right)^2 + \left(\frac{4}{5}\right)^2} = \sqrt{\frac{9+16}{25}} = 1\). ✓

Common Mistakes

✕ Forgetting the Square Root
Computing \(v_1^2 + v_2^2\) but stopping there. The \(\ell^2\) norm requires the final \(\sqrt{\cdot}\).
✕ Forgetting Absolute Values
In the \(\ell^1\) norm, negative components must be made positive first: \(|-3| = 3\), not \(-3\).
✕ Confusing Norm with Dot Product
\(\|\mathbf{v}\|^2 = \mathbf{v} \cdot \mathbf{v}\), but \(\|\mathbf{v}\| \neq \mathbf{v} \cdot \mathbf{v}\). Don't forget the square root.

Norm Summary Table

Property \(\ell^1\) \(\ell^2\) \(\ell^\infty\)
Formula \(\sum |v_i|\) \(\sqrt{\sum v_i^2}\) \(\max |v_i|\)
Unit Ball Shape Diamond Circle / Sphere Square / Cube
Also Called Manhattan / Taxicab Euclidean Chebyshev / Max
\(\mathbf{v} = (1,-2,2)\) 5 3 2
Ordering \(\|\mathbf{v}\|_\infty \leq \|\mathbf{v}\|_2 \leq \|\mathbf{v}\|_1\)

Question 1 of 5

What is the Euclidean norm of the vector \(\mathbf{v} = (6, 8)\)?

Question 2 of 5

What is the distance between \(\mathbf{p} = (2, 3)\) and \(\mathbf{q} = (5, 7)\)?

Question 3 of 5

For \(\mathbf{w} = (-3, 4, 0)\), what is \(\|\mathbf{w}\|_\infty\)?

Question 4 of 5

If \(\mathbf{u} = (2, -1)\), what is the unit vector \(\hat{\mathbf{u}}\)?

Question 5 of 5

Which property states that \(\|\mathbf{u} + \mathbf{v}\| \leq \|\mathbf{u}\| + \|\mathbf{v}\|\)?