Vector Norms & Distance — Vector Norms and Distance - Linear Algebra

SECTION 1

Historical Context & Motivation

Have you ever wondered how your phone's GPS knows exactly how far you are from a restaurant? Or how a video game calculates whether an enemy is close enough to attack? These questions all come down to one idea: measuring distance. Mathematicians have spent centuries figuring out how to measure length and distance not just on a flat surface, but in spaces with many dimensions.

The story of vector norms (a fancy name for "the length of a vector") stretches back to ancient Greece and continues through modern computer science. Let's trace how this idea developed.

~300 BC • Euclid's Geometry

The Greek mathematician Euclid wrote Elements, which included the Pythagorean theorem — the foundation for measuring distance between two points on a flat surface.

1637 • Descartes Introduces Coordinates

René Descartes created the coordinate system (the x-y plane you use in class), allowing distance to be calculated using algebra instead of just drawings.

1880s • Minkowski & Higher Dimensions

Hermann Minkowski generalized distance formulas beyond the familiar Euclidean version. He introduced the idea of different "norms" — different ways to measure length — which became essential in physics and math.

1922 • Banach's Normed Spaces

Stefan Banach formally defined normed vector spaces, giving mathematicians a rigorous framework to study length and distance in abstract settings.

2000s+ • Machine Learning & Data Science

Today, vector norms and distance are everywhere — from facial recognition to recommendation engines. Choosing the right norm can determine whether an algorithm works well or fails.

So here is the central question: How do we measure the "size" of a vector, and how do we use that measurement to find the distance between two points? That's exactly what this lesson will answer.

SECTION 2

Core Principles & Definitions

Before we dive into formulas, let's nail down some key ideas. A vector is a quantity that has both a direction and a magnitude (size). You can think of it as an arrow pointing from one spot to another. A norm is simply a rule that assigns a non-negative number to a vector — that number tells you the vector's length.

What Is a Norm?

A norm is a function that takes a vector and returns a single number representing its "length." It is always zero or positive.

Non-Negativity

A norm is never negative. The only vector with a norm of zero is the zero vector (the vector with all components equal to 0).

Scalar Multiplication

If you multiply a vector by a number c, the norm gets multiplied by |c|. Doubling a vector doubles its length.

Triangle Inequality

The norm of the sum of two vectors is always less than or equal to the sum of their individual norms. In plain English: a shortcut is never longer than going the long way around.

Distance = Norm of the Difference

The distance between two vectors u and v is defined as the norm of their difference: ‖u − v‖.

✦

KEY TAKEAWAY

Think of a norm like a ruler for arrows. Just as you can measure how long a pencil is with a ruler, a norm measures how long a vector is. And if you want to know how far apart two arrows' tips are, you subtract one vector from the other and then measure the result — that gives you the distance.

SECTION 3

Visual Explanation

Let's see what vector norms and distance look like on a coordinate plane. The diagram below shows two vectors, u = (3, 4) and v = (7, 1). Each vector's norm is its length from the origin, and the distance between them is the length of the dashed line connecting their tips.

The purple arrow is vector u = (3, 4) with norm 5. The cyan arrow is vector v = (7, 1) with norm ≈ 7.07. The dashed gold line shows the distance between their endpoints.

Notice that the length of the purple arrow (‖u‖ = 5) comes from the Pythagorean theorem: √(3² + 4²) = √(9 + 16) = √25 = 5. That's all a norm really is — a generalization of the Pythagorean theorem to measure vector length. The distance between u and v is the norm of the difference vector u − v = (3 − 7, 4 − 1) = (−4, 3), which gives ‖u − v‖ = √(16 + 9) = 5.

SECTION 4

Mathematical Framework

Now let's formalize the math. There are several different norms, but the most common ones you'll encounter are the L¹ norm (also called the Manhattan norm), the L² norm (also called the Euclidean norm), and the L∞ norm (also called the max norm). Each one measures "length" in a slightly different way.

EUCLIDEAN NORM (L² NORM)

‖v‖₂ = √(v₁² + v₂² + … + vₙ²)

This is the most familiar norm — it gives the straight-line distance from the origin to the tip of the vector. Here v₁, v₂, …, vₙ are the components of the vector v.

MANHATTAN NORM (L¹ NORM)

‖v‖₁ = |v₁| + |v₂| + … + |vₙ|

Add up the absolute values of all components. It's called the "Manhattan" norm because it measures distance the way you'd walk on a city grid — only horizontal and vertical moves, no cutting diagonally.

MAX NORM (L∞ NORM)

‖v‖∞ = max(|v₁|, |v₂|, …, |vₙ|)

Take the largest absolute value among all components. This norm only cares about the single biggest entry in the vector.

DISTANCE FORMULA

d(u, v) = ‖u − v‖

The distance between two vectors u and v is always the norm of their difference. Whichever norm you choose (L¹, L², or L∞), the distance formula stays the same — just plug in the appropriate norm.

SECTION 5

Comparing Different Norms

Different norms give different answers for the same vector. The diagram below shows the unit circles for the L¹, L², and L∞ norms. A "unit circle" for a norm is the set of all vectors whose norm equals exactly 1. Each norm defines a different shape.

The L¹ unit circle (diamond) is rotated 45°. The L² unit circle is the familiar round circle. The L∞ unit circle (square) captures all vectors whose largest component is at most 1.

Let's look at a specific example. For the vector v = (3, −4), the three norms give different values.

Norm	Formula	Calculation for v = (3, −4)	Result
L¹	\|v₁\| + \|v₂\|	\|3\| + \|−4\| = 3 + 4	7
L²	√(v₁² + v₂²)	√(9 + 16) = √25	5
L∞	max(\|v₁\|, \|v₂\|)	max(3, 4)	4

Comparing L¹, L², and L∞ norms for the vector (3, −4)

💡

Notice the Pattern

For any vector, the L∞ norm is always the smallest, the L¹ norm is always the largest, and the L² norm falls in between: ‖v‖∞ ≤ ‖v‖₂ ≤ ‖v‖₁. This makes sense when you look at the unit circle diagram — the square (L∞) contains the circle (L²), which contains the diamond (L¹).

SECTION 6

Worked Example

Let's work through a complete problem. Suppose you have two vectors in three-dimensional space: u = (1, −2, 3) and v = (4, 0, −1). Find the Euclidean (L²) distance between u and v.

Finding the Euclidean Distance Between Two 3D Vectors

1
Step 1 — Find the Difference Vector

Subtract corresponding components: u − v = (1 − 4, −2 − 0, 3 − (−1)) = (1 − 4, −2 − 0, 3 + 1).

u − v = (−3, −2, 4)
2
Step 2 — Square Each Component

Square each entry of the difference vector: (−3)² = 9, (−2)² = 4, and 4² = 16.

Squares: 9, 4, 16
3
Step 3 — Sum the Squares

Add up all the squared values: 9 + 4 + 16 = 29.

Sum = 29
4
Step 4 — Take the Square Root

The Euclidean distance is the square root of the sum: √29 ≈ 5.39.

d(u, v) = ‖u − v‖₂ = √29 ≈ 5.39

⚠

Pro Tip

Always be careful with signs! When you subtract vectors, a double negative (like 3 − (−1)) becomes addition. This is one of the most common places students make mistakes.

SECTION 7

When to Use Each Norm

You might wonder: if the L² norm is the "natural" one, why bother with the others? It turns out that different situations call for different norms. Each norm captures a different idea of "closeness," and choosing the right one can make a big difference in real-world applications.

Norm	Strengths	Limitations	Best Used For
L¹ (Manhattan)	Robust to outliers (extreme values). Easy to compute.	Doesn't give the shortest path geometrically.	City-block navigation, sparse data analysis, machine learning feature selection.
L² (Euclidean)	Matches physical intuition. Smooth and differentiable everywhere.	Sensitive to outliers due to squaring. Computationally heavier in high dimensions.	Physics calculations, GPS distance, k-nearest neighbor algorithms.
L∞ (Max)	Fastest to compute. Focuses on the worst-case component.	Ignores all components except the largest, losing information.	Error checking (worst-case analysis), chess-board distance, quality control.

Comparing the three most common vector norms

✦

KEY TAKEAWAY

Choosing a norm is like choosing how to grade a test. The L¹ norm is like adding up all the point deductions — every mistake counts equally. The L² norm is like combining all errors into one "overall" score — big mistakes count more because of the squaring. The L∞ norm is like saying "your grade is determined by your worst single answer." Different classes might use different approaches, and each gives a different picture of performance.

SECTION 8

Connection to Advanced Theory

The concepts of norms and distance you've learned here are the foundation for much more advanced math. In college-level linear algebra, you'll encounter inner product spaces — places where the L² norm comes from a special operation called the inner product (also known as the dot product). The relationship is: ‖v‖₂ = √(v · v), where v · v is the dot product of the vector with itself.

What You Learned Today	Where It Leads
Norm of a vector (‖v‖)	Normed vector spaces, Banach spaces — abstract spaces where length is defined.
Euclidean distance d(u, v) = ‖u − v‖₂	Metric spaces — any space with a distance function obeying specific rules.
Different norms (L¹, L², L∞)	Lp norms for any p ≥ 1, function spaces, and optimization theory.
Triangle inequality	Cauchy-Schwarz inequality, orthogonality, and projection theorems.

From today's concepts to advanced linear algebra

Don't worry if those advanced terms sound intimidating right now! The important thing is that you already understand the core idea: norms measure length, and distance is the norm of the difference between two vectors. Every advanced topic in this area builds on exactly that foundation.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

In your own words, explain what a vector norm measures. Why must a norm always be non-negative?

Show answer

A vector norm measures the "length" or "size" of a vector — how far the tip of the vector is from the origin. It must always be non-negative because length is a physical quantity that can't be negative. You can't have a ruler that reads −3 centimeters. The only vector with a norm of zero is the zero vector, which has no length because it goes nowhere. This property ensures that the norm behaves like a sensible measurement system.

Question 2: BASIC CALCULATION

Find the Euclidean norm (L²) of the vector w = (5, −12).

Show answer

Apply the formula ‖w‖₂ = √(w₁² + w₂²). Substitute: ‖w‖₂ = √(5² + (−12)²) = √(25 + 144) = √169 = 13. The Euclidean norm of (5, −12) is exactly 13. You may recognize this as a 5-12-13 right triangle — one of the classic Pythagorean triples!

Question 3: INTERMEDIATE

Given u = (2, −1, 3) and v = (−1, 4, 1), find the distance between u and v using the L¹ (Manhattan) norm.

Show answer

First, find the difference vector: u − v = (2 − (−1), −1 − 4, 3 − 1) = (3, −5, 2). Now apply the L¹ norm: ‖u − v‖₁ = |3| + |−5| + |2| = 3 + 5 + 2 = 10. The Manhattan distance between u and v is 10. Notice this is larger than the Euclidean distance would be (√(9 + 25 + 4) = √38 ≈ 6.16), which makes sense because the L¹ norm always gives a result greater than or equal to the L² norm.

Question 4: APPLIED

A drone is at position (10, 20, 50) in meters, and its landing pad is at (30, 5, 0). Find the straight-line distance (Euclidean distance) between the drone and the pad. Then find the L∞ distance and explain what it represents physically.

Show answer

Difference vector: (10 − 30, 20 − 5, 50 − 0) = (−20, 15, 50). Euclidean distance: ‖d‖₂ = √(400 + 225 + 2500) = √3125 = 25√5 ≈ 55.9 meters. This is the straight-line distance through the air. For the L∞ distance: ‖d‖∞ = max(|−20|, |15|, |50|) = 50 meters. The L∞ distance tells us that the largest gap in any single direction is 50 meters — specifically, the altitude difference. This could be useful if we want to know the minimum range the drone's altimeter must handle.

Question 5: CRITICAL THINKING

Prove that the L∞ norm of any vector in ℝⁿ is always less than or equal to its L¹ norm. That is, show ‖v‖∞ ≤ ‖v‖₁ for any vector v. Use the definitions of both norms in your argument.

Show answer

Let v = (v₁, v₂, …, vₙ). By definition, ‖v‖∞ = max(|v₁|, |v₂|, …, |vₙ|), and ‖v‖₁ = |v₁| + |v₂| + … + |vₙ|. Let's say the maximum absolute value occurs at component k, so ‖v‖∞ = |vₖ|. Since |vₖ| is just one of the terms in the sum |v₁| + |v₂| + … + |vₙ|, and all absolute values are non-negative (so adding more terms can only increase or maintain the total), we get |vₖ| ≤ |v₁| + |v₂| + … + |vₙ|. Therefore ‖v‖∞ ≤ ‖v‖₁. Equality holds only when all but one component is zero.

SUMMARY

Lesson Summary

Summary

A vector norm is a function that assigns a non-negative number to a vector, representing its length. The three most important norms are the L¹ (Manhattan) norm, which sums the absolute values of all components; the L² (Euclidean) norm, which uses the Pythagorean theorem to find straight-line length; and the L∞ (max) norm, which returns the largest absolute component. Every valid norm satisfies three properties: non-negativity, scalar multiplication, and the triangle inequality.

The distance between two vectors is always defined as the norm of their difference: d(u, v) = ‖u − v‖. Different norms produce different distance values for the same pair of vectors, and the choice of norm depends on the application. The unit circle for each norm has a distinct shape — a diamond for L¹, a circle for L², and a square for L∞ — which visually captures how each norm "sees" the world. These ideas form the backbone of inner product spaces and orthogonality, two of the most powerful concepts in advanced linear algebra.