Orthonormal Bases - Linear Algebra

SECTION 1

Historical Context & Motivation

Imagine trying to give someone directions in a city where the streets cross at weird angles instead of clean right angles. Every instruction would be confusing! Mathematicians faced a similar challenge when they worked with coordinate systems built from vectors that weren't perpendicular. The search for the cleanest, most efficient way to describe space led to the idea of orthonormal bases — special sets of direction vectors that are all perpendicular to one another and each exactly one unit long.

This concept didn't appear overnight. It grew over centuries as mathematicians and physicists needed better tools to describe geometry, solve equations, and analyze signals. Let's look at some key moments in that story.

1637 • Descartes Introduces Coordinates

René Descartes published his coordinate system, giving us the familiar x-axis and y-axis. These two axes are perpendicular and laid the groundwork for describing space with numbers.

1843 • Hamilton & Higher Dimensions

William Rowan Hamilton invented quaternions, extending the idea of perpendicular axes to higher dimensions. Mathematicians began wondering how to generalize perpendicularity beyond 3D.

1907 • Schmidt's Orthogonalization Process

Erhard Schmidt published a step-by-step method (now called the Gram–Schmidt process) for turning any set of vectors into an orthonormal set. This made orthonormal bases practical to build.

1932 • Von Neumann Formalizes Hilbert Spaces

John von Neumann showed that orthonormal bases are essential in quantum mechanics. His work connected abstract math to the physical world, cementing the concept's importance.

1965 • FFT & Digital Signal Processing

The Fast Fourier Transform algorithm used orthonormal sine and cosine waves to analyze sound and images. Today, every digital photo and music file relies on orthonormal ideas.

Throughout history, the central question has stayed the same: How can we choose direction vectors so that breaking a vector into components is as simple as possible? Orthonormal bases are the answer.

SECTION 2

Core Principles & Definitions

Before we build an orthonormal basis, we need to understand three ideas that stack on top of each other like building blocks. Each idea adds one more rule to our set of vectors.

Basis

A basis is a set of vectors that can be combined (using addition and scaling) to reach every point in a space. Think of them as the minimum set of direction arrows you need.

Orthogonal

Two vectors are orthogonal when they are perpendicular — they meet at a 90° angle. Their dot product equals zero. An orthogonal basis is a basis where every pair of vectors is perpendicular.

Orthonormal

An orthonormal basis goes one step further: not only are all vectors perpendicular, but each vector also has a length (magnitude) of exactly 1. "Ortho" means perpendicular; "normal" means unit length.

Inner Product (Dot Product)

The inner product (often called the dot product) measures how much two vectors point in the same direction. For orthonormal vectors, the dot product is 0 between different vectors and 1 with themselves.

✦

KEY TAKEAWAY

Think of an orthonormal basis like the perfectly square grid on a sheet of graph paper. Each axis points in its own direction (perpendicular), and each square is exactly one unit wide (unit length). If the grid lines were tilted or different lengths, reading coordinates would be a headache. An orthonormal basis gives you the cleanest possible graph paper for any dimension.

A handy way to remember the two requirements is the phrase "perpendicular and unit-length." If any pair of basis vectors isn't perpendicular, or if any vector's length isn't exactly 1, the set is not orthonormal.

SECTION 3

Visual Explanation

The diagram below shows three different bases for two-dimensional space. Compare them side by side to see why orthonormal is the gold standard.

The left panel shows a general basis with vectors at an awkward angle and unequal lengths. The middle panel improves by making the vectors perpendicular, but their lengths still differ. The right panel shows the orthonormal basis: both vectors are perpendicular and sit exactly on the unit circle (dashed), meaning each has length 1.

Notice the dashed unit circle in the right panel. Both green and orange arrows end exactly on that circle, confirming their length is 1. The small square at the origin marks the 90° angle between them. When both conditions — perpendicular and unit length — are satisfied, we say the basis is orthonormal.

SECTION 4

Mathematical Framework

Let's translate the visual ideas into formulas. We'll use the dot product to test perpendicularity and length. If you've seen the dot product before, great — if not, it's just a way to multiply two vectors and get a single number.

DOT PRODUCT (INNER PRODUCT)

u⃗ · v⃗ = u₁v₁ + u₂v₂ + … + uₙvₙ

Multiply matching components and add them up. If u⃗ = (2, 3) and v⃗ = (4, −1), then u⃗ · v⃗ = 2 × 4 + 3 × (−1) = 5.

ORTHONORMALITY CONDITIONS

eᵢ · eⱼ = δᵢⱼ (equals 1 if i = j, equals 0 if i ≠ j)

The symbol δᵢⱼ is called the Kronecker delta. It's a shorthand: when you dot a basis vector with itself you get 1 (unit length), and when you dot it with a different basis vector you get 0 (perpendicular).

EASY COORDINATE FORMULA

cᵢ = v⃗ · eᵢ

To find how much of basis vector eᵢ is in any vector v⃗, just take their dot product. No messy system of equations needed! This only works when the basis is orthonormal.

RECONSTRUCTION OF A VECTOR

v⃗ = (v⃗ · e₁)e₁ + (v⃗ · e₂)e₂ + … + (v⃗ · eₙ)eₙ

Once you know each coefficient cᵢ, you rebuild the original vector by scaling each basis vector by its coefficient and adding them all together.

💡

Why does this matter?

With a non-orthonormal basis, finding coordinates requires solving a system of equations (matrices, row reduction, etc.). With an orthonormal basis, you just compute dot products — one for each coordinate. That's a huge time saver, especially in higher dimensions.

SECTION 5

Building an Orthonormal Basis: Gram–Schmidt Process

How do you actually create an orthonormal basis? The most famous method is the Gram–Schmidt process. It takes any set of linearly independent vectors and systematically converts them into an orthonormal set. Think of it like straightening crooked fence posts one at a time and then cutting each to the same height.

The Gram–Schmidt process in two dimensions. Step 1: Normalize the first vector to get e₁. Step 2: Subtract the component of v₂ that lies along e₁ (the projection), leaving a perpendicular remainder w₂, then normalize it to get e₂.

Step 1 — Normalize: Divide v₁ by its length to create the first unit vector e₁.
Step 2 — Subtract the projection: Compute how much of v₂ points along e₁, then subtract that piece. What's left (w₂) is perpendicular to e₁.
Step 3 — Normalize again: Divide w₂ by its length to get the second unit vector e₂.
Repeat: For each additional vector, subtract projections onto all previously found e vectors, then normalize. This extends to 3D, 4D, or any dimension.

SECTION 6

Worked Example

Let's walk through a complete example. We'll start with two vectors that form a basis for 2D space but aren't orthonormal, and we'll apply the Gram–Schmidt process to fix them.

Convert {v₁, v₂} into an orthonormal basis

1
Step 1 — State the given vectors

We are given v₁ = (3, 4) and v₂ = (2, 1). These are linearly independent (neither is a scaled copy of the other), so they form a basis — but they are not orthonormal.
2
Step 2 — Find e₁ by normalizing v₁

First, compute the length of v₁: ‖v₁‖ = √(3² + 4²) = √(9 + 16) = √25 = 5. Now divide each component by 5:

e₁ = v₁ / ‖v₁‖ = (3/5, 4/5) = (0.6, 0.8)
3
Step 3 — Compute the projection of v₂ onto e₁

The projection coefficient is v₂ · e₁ = 2 × 0.6 + 1 × 0.8 = 1.2 + 0.8 = 2.0. So the projection vector is 2.0 × e₁ = 2.0 × (0.6, 0.8) = (1.2, 1.6).

proj = (1.2, 1.6)
4
Step 4 — Subtract the projection to get w₂

w₂ = v₂ − proj = (2, 1) − (1.2, 1.6) = (0.8, −0.6). This vector is perpendicular to e₁. You can verify: w₂ · e₁ = 0.8 × 0.6 + (−0.6) × 0.8 = 0.48 − 0.48 = 0. ✓

w₂ = (0.8, −0.6)
5
Step 5 — Normalize w₂ to get e₂

‖w₂‖ = √(0.8² + (−0.6)²) = √(0.64 + 0.36) = √1 = 1. Lucky — w₂ is already unit length! So e₂ = (0.8, −0.6).

e₂ = (0.8, −0.6)
6
Step 6 — State the orthonormal basis

Our orthonormal basis is {e₁, e₂} = {(0.6, 0.8), (0.8, −0.6)}. Let's verify: e₁ · e₂ = 0.6 × 0.8 + 0.8 × (−0.6) = 0.48 − 0.48 = 0 (perpendicular ✓). ‖e₁‖ = 1 ✓. ‖e₂‖ = 1 ✓.

Orthonormal basis: {(0.6, 0.8), (0.8, −0.6)}

SECTION 7

Orthonormal vs. Other Bases

Orthonormal bases aren't the only kind of basis, so when should you use them? The table below compares three types of bases across the properties that matter most.

Property	General Basis	Orthogonal Basis	Orthonormal Basis
Vectors perpendicular?	No	Yes ✓	Yes ✓
Vectors unit length?	No	No	Yes ✓
Finding coordinates	Solve a system of equations	Dot product ÷ length²	Just a dot product!
Length preserved?	Not guaranteed	Needs adjustment	Yes — Parseval's identity
Ease of use	★☆☆	★★☆	★★★

Comparison of basis types

✦

KEY TAKEAWAY

Choosing an orthonormal basis is like switching from a wobbly, hand-drawn map to a GPS with a perfect grid. Both can tell you where you are, but the GPS makes navigation effortless. Whenever you have the freedom to pick your basis, pick an orthonormal one — your calculations will thank you.

SECTION 8

Connection to Advanced Topics

Orthonormal bases are a gateway to many powerful ideas in math and science. Once you're comfortable with them in 2D and 3D, you'll encounter them in surprising places.

Concept in This Lesson	Where It Leads
Dot product = 0 means perpendicular	Generalized inner products in function spaces (Fourier analysis)
Gram–Schmidt process	QR decomposition — a core algorithm in data science and machine learning
cᵢ = v⃗ · eᵢ (easy coordinates)	Fourier coefficients — breaking sound into frequencies
Orthonormal basis in ℝⁿ	Orthonormal bases in infinite-dimensional Hilbert spaces (quantum mechanics)

From orthonormal bases to advanced math

In Fourier analysis, sine and cosine waves of different frequencies form an orthonormal basis for the space of periodic functions. When you listen to music on a streaming service, the audio file uses these ideas to compress sound efficiently. In quantum mechanics, the possible states of a particle live in an abstract space, and measurements correspond to projecting onto orthonormal basis vectors. The math you've learned in this lesson is the same math that powers those advanced applications.

ℹ

Looking Ahead

If you continue with linear algebra, the next big topic after orthonormal bases is eigenvalues and eigenvectors. Many important matrices have orthonormal sets of eigenvectors, which makes diagonalization especially clean — another win for orthonormality!

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

A student claims that the set {(1, 0), (0, 2)} is an orthonormal basis for ℝ². Is the student correct? Explain why or why not.

Show answer

The student is incorrect. While the two vectors are perpendicular (their dot product is 1 × 0 + 0 × 2 = 0), the second vector (0, 2) has a length of 2, not 1. An orthonormal basis requires every vector to be a unit vector (length 1). The corrected set would be {(1, 0), (0, 1)}, where (0, 2) is divided by its length.

Question 2: BASIC CALCULATION

Given the orthonormal basis {e₁ = (1, 0), e₂ = (0, 1)} and the vector v⃗ = (7, −3), find the coordinates of v⃗ in this basis using dot products.

Show answer

Using the formula cᵢ = v⃗ · eᵢ: c₁ = v⃗ · e₁ = 7 × 1 + (−3) × 0 = 7. c₂ = v⃗ · e₂ = 7 × 0 + (−3) × 1 = −3. So the coordinates are (7, −3). In this case the standard basis matches the orthonormal basis, so the coordinates look the same as the components — but the key insight is that dot products gave us the answer instantly without solving equations.

Question 3: INTERMEDIATE

Use the Gram–Schmidt process to convert v₁ = (1, 1) and v₂ = (1, −1) into an orthonormal basis for ℝ². Verify your answer.

Show answer

Step 1: Normalize v₁. ‖v₁‖ = √(1² + 1²) = √2. So e₁ = (1/√2, 1/√2). Step 2: Compute v₂ · e₁ = 1 × (1/√2) + (−1) × (1/√2) = 0. The projection is 0 × e₁ = (0, 0). This means v₂ is already perpendicular to e₁! Step 3: Normalize v₂. ‖v₂‖ = √(1 + 1) = √2. So e₂ = (1/√2, −1/√2). Verification: e₁ · e₂ = (1/√2)(1/√2) + (1/√2)(−1/√2) = 1/2 − 1/2 = 0 ✓. ‖e₁‖ = 1 ✓. ‖e₂‖ = 1 ✓. The orthonormal basis is {(1/√2, 1/√2), (1/√2, −1/√2)}.

Question 4: APPLIED

A game developer uses the orthonormal basis {e₁ = (0.6, 0.8), e₂ = (0.8, −0.6)} to describe a rotated coordinate system. A character's position is v⃗ = (5, 5). Find the character's coordinates in the rotated system.

Show answer

Using the orthonormal coordinate formula cᵢ = v⃗ · eᵢ: c₁ = 5 × 0.6 + 5 × 0.8 = 3.0 + 4.0 = 7.0. c₂ = 5 × 0.8 + 5 × (−0.6) = 4.0 − 3.0 = 1.0. So in the rotated system, the character is at (7, 1). Notice how simple this was — just two dot products. If the basis were not orthonormal, we'd need to set up and solve a 2×2 system of linear equations instead. This is exactly why game engines prefer orthonormal bases for camera and character coordinate frames.

Question 5: CRITICAL THINKING

Suppose you have three vectors in 3D: v₁ = (1, 0, 0), v₂ = (1, 1, 0), and v₃ = (1, 1, 1). Without performing all the calculations, describe the strategy you'd use to turn them into an orthonormal basis. Then explain: could you start with v₃ instead of v₁? Would the final orthonormal basis be the same?

Show answer

Strategy: Apply Gram–Schmidt in order. First, normalize v₁ to get e₁. Then subtract the projection of v₂ onto e₁, and normalize the result to get e₂. Finally, subtract the projections of v₃ onto both e₁ and e₂, and normalize what remains to get e₃. At each step, the new vector is made perpendicular to all previous basis vectors. If you started with v₃ instead, the process would still produce a valid orthonormal basis, but it would generally be a different one. The Gram–Schmidt process depends on the order of input vectors. Both results are legitimate orthonormal bases for ℝ³ — there are infinitely many orthonormal bases for any space. The key property (orthonormality) is guaranteed regardless of starting order.

SUMMARY

Lesson Summary

Summary

An orthonormal basis is a set of vectors that are all mutually perpendicular (orthogonal) and each has a length of exactly 1 (normalized). The mathematical test is eᵢ · eⱼ = δᵢⱼ — dot any basis vector with itself and get 1, dot it with a different one and get 0. The biggest payoff is the easy coordinate formula cᵢ = v⃗ · eᵢ, which lets you find components with simple dot products instead of solving systems of equations.

You can build an orthonormal basis from any set of independent vectors using the Gram–Schmidt process: normalize the first vector, subtract projections to make each subsequent vector perpendicular, then normalize it. Orthonormal bases appear everywhere — from computer graphics and signal processing to quantum mechanics. Mastering them gives you a powerful toolkit for any future work in linear algebra and beyond.