Projections onto Lines & Subspaces — Projections onto Lines and Subspaces (Intro) - Linear Algebra

SECTION 1

Historical Context & Motivation

Imagine you are standing in a field and you want to find the spot on a straight road that is closest to you. You would walk straight toward the road at a right angle — that is exactly what a projection does in math. Projections help us drop a vector down onto a line or a flat surface (called a subspace) so we can find the closest point on that line or surface.

Mathematicians have been thinking about this idea for centuries. The concept of projecting one thing onto another appears in geometry, physics, engineering, and even computer graphics. Here are some key moments that shaped how we understand projections today.

~300 BC • Euclid's Perpendicular Lines

Euclid studied perpendicular lines in his famous work Elements. He showed that the shortest distance from a point to a line is always the perpendicular (right-angle) path. This is the geometric root of projection.

1637 • Descartes Introduces Coordinates

René Descartes invented the coordinate plane, letting mathematicians describe points and lines with numbers. This made it possible to compute projections using algebra instead of drawing pictures.

1843 • Hamilton & Vector Algebra

William Rowan Hamilton developed new ways to work with vectors. His ideas helped formalize the dot product, which is the key tool used in the projection formula.

1900s • Modern Linear Algebra

Mathematicians built linear algebra into a complete framework for studying spaces and transformations. Projections became a central idea in fields like data science, machine learning, and signal processing.

The big question that projections answer is: given a vector and a line (or subspace), what is the closest point on that line to the tip of the vector? This turns out to be one of the most useful ideas in all of mathematics.

SECTION 2

Core Principles & Definitions

Before we dive into formulas, let's nail down the key ideas that make projections work. Think of these as the building blocks you need to understand first.

Vectors

A vector is a quantity with both direction and length (magnitude). You can picture it as an arrow pointing from the origin to a specific point.

Dot Product

The dot product of two vectors measures how much they point in the same direction. It multiplies matching components and adds the results: a₁b₁ + a₂b₂.

Perpendicularity

Two vectors are perpendicular (orthogonal) when they meet at a 90° angle. The dot product of perpendicular vectors is always zero.

Subspace

A subspace is a flat region that passes through the origin — like a line through the origin or a plane through the origin. It's the 'target' we project onto.

Projection

A projection takes a vector and finds the closest point to it on a given line or subspace. The error (leftover part) is always perpendicular to the target.

✦

KEY TAKEAWAY

Think of projection like a shadow. When the sun is directly above you, your shadow on the ground is the projection of your body onto the flat ground. The sunlight travels straight down (perpendicular to the ground), and your shadow is the closest flat version of you. In math, the projection of a vector onto a line works the same way — you drop it straight down at a right angle to get the closest point on the line.

SECTION 3

Visual Explanation

The diagram below shows a vector b being projected onto a line defined by the direction vector a. The projection (labeled proj_ab) lands on the line, and the error vector e is perpendicular to the line.

The cyan arrow is the direction vector a defining the line. The pink arrow is the original vector b. The amber arrow is the projection (the closest point on the line), and the dashed green arrow is the error. Notice the right-angle symbol where the error meets the line.

The key insight from this diagram is that the error vector e is always perpendicular to the line. This is what makes the projection the closest point. If the error were tilted even slightly, you could find a closer point — but when the error is perfectly perpendicular, you have found the minimum distance.

SECTION 4

Mathematical Framework

Now let's build the formula step by step. We want to project vector b onto the line defined by vector a. The result is a new vector that lies on the line through a and is as close as possible to b.

PROJECTION ONTO A LINE

proj_a(b) = (a · b / a · a) × a

a · b is the dot product of vectors a and b. a · a is the dot product of a with itself (which equals the length of a squared, ‖a‖²). The fraction (a · b)/(a · a) is a single number (scalar) that tells you how many copies of a you need.

Here is why this works. The projection must be some scaled version of a, because it has to lie on the line through a. So we write proj_a(b) = c × a, where c is some number we need to find. The error vector e = b − c × a must be perpendicular to a. That means a · e = 0.

PERPENDICULARITY CONDITION

a · (b − c × a) = 0

Expanding: a · b − c × (a · a) = 0, so c = (a · b) / (a · a). This is the scalar coefficient that determines the projection.

ERROR VECTOR

e = b − proj_a(b)

The error vector e is the part of b that is not captured by the projection. It is always perpendicular to the line, meaning a · e = 0.

💡

Remember

The dot product a · b = a₁b₁ + a₂b₂ (in 2D) or a₁b₁ + a₂b₂ + a₃b₃ (in 3D). It gives you a single number, not a vector. You then multiply that number by the vector a to get the projection vector.

SECTION 5

Detailed Breakdown — From Lines to Subspaces

So far we have looked at projecting onto a single line. But what if your target is a whole plane or a higher-dimensional flat surface? That is called projecting onto a subspace. The idea is exactly the same — find the point on the subspace that is closest to your vector — but the formula uses matrices instead of single vectors.

Left: projecting vector b onto a line (1D subspace). Right: projecting vector b onto a plane (2D subspace). In both cases, the green dashed error vector is perpendicular to the target.

When projecting onto a line, your target is one-dimensional — it is defined by a single direction vector a. When projecting onto a plane, your target is two-dimensional — it is defined by two direction vectors. The principle stays the same: the error is always perpendicular to the target space.

Feature	Projection onto a Line	Projection onto a Subspace
Target	A single direction vector a	A set of direction vectors (columns of matrix A)
Dimension	1D	2D, 3D, or higher
Formula uses	Dot products only	Matrix multiplication and inverses
Error perpendicular to	The line	The entire subspace

Comparison of line projections vs. subspace projections

SECTION 6

Worked Example

Let's work through a concrete example. We will project vector b = (3, 4) onto the line defined by vector a = (1, 2).

Project b = (3, 4) onto the line through a = (1, 2)

1
Step 1 — Compute the dot product a · b

Multiply matching components and add: a · b = (1)(3) + (2)(4) = 3 + 8 = 11.

a · b = 11
2
Step 2 — Compute the dot product a · a

This gives us the squared length of a: a · a = (1)(1) + (2)(2) = 1 + 4 = 5.

a · a = 5
3
Step 3 — Find the scalar coefficient

Divide the two dot products: c = (a · b) / (a · a) = 11 / 5 = 2.2. This tells us the projection is 2.2 copies of vector a.

c = 11/5 = 2.2
4
Step 4 — Compute the projection vector

Multiply the scalar by vector a: proj_a(b) = 2.2 × (1, 2) = (2.2, 4.4). You can also write this as (11/5, 22/5).

proj_a(b) = (11/5, 22/5)
5
Step 5 — Compute and verify the error

e = b − proj = (3, 4) − (2.2, 4.4) = (0.8, −0.4). Check: a · e = (1)(0.8) + (2)(−0.4) = 0.8 − 0.8 = 0. The error is indeed perpendicular to a. ✓

e = (0.8, −0.4), and a · e = 0 ✓

💡

Verification Tip

Always check your answer by computing a · e. If the dot product is zero, your projection is correct. This is a built-in error check that works every time!

SECTION 7

Strengths, Limitations & Common Mistakes

Projections are powerful tools, but there are some important things to watch out for. Let's look at what projections do well and where students sometimes get tripped up.

Strengths	Limitations / Common Mistakes
The formula is straightforward — just dot products and scalar multiplication.	You cannot project onto a vector of length zero (the zero vector). Division by zero would occur in a · a.
The perpendicularity check (a · e = 0) gives you a reliable way to verify your work.	Students sometimes confuse the scalar coefficient c with the projection vector. Remember: c is just a number; the projection is c × a.
Projections work in any number of dimensions — 2D, 3D, 100D, and beyond.	The line must pass through the origin. If it does not, you need to shift coordinates first.
Projections are the foundation for least-squares fitting, which is used everywhere from science to economics.	Projecting onto a subspace (not just a line) requires matrix operations, which are more advanced.

Strengths and common pitfalls of projection

✦

KEY TAKEAWAY

Projections are like GPS navigation for vectors. Just as GPS finds the closest point on a road to your current location, projection finds the closest point on a line or subspace to your vector. The "error" is the distance you would have to travel from the road to reach your actual position, and it is always a straight (perpendicular) path.

SECTION 8

Connection to Advanced Theory

The projection formula you learned here is the starting point for some incredibly powerful advanced topics. As you continue studying linear algebra, you will see projections appear again and again. Here is a preview of where this idea leads.

What You Learned	Where It Leads
Projection onto a line using the dot product	Projection matrices: P = A(AᵀA)⁻¹Aᵀ, which handle projections onto any subspace with a single matrix multiplication
The error vector is perpendicular to the line	Orthogonal decomposition: any vector can be split into a part inside a subspace and a part perpendicular to it
Finding the closest point on a line	Least-squares approximation: finding the best-fit line through data points when no exact solution exists
Scalar coefficient c = (a · b)/(a · a)	Fourier series: breaking complex signals into simple waves using projections onto sine and cosine functions

How today's concepts connect to advanced topics

The beauty of projections is that the core idea never changes — find the closest point and make sure the error is perpendicular. Whether you are working with two-component vectors in a classroom or million-dimensional data in a machine learning algorithm, this same principle applies.

SECTION 9

Practice Problems

Question 1: CONCEPTUAL

In your own words, explain why the error vector in a projection must be perpendicular to the target line. What would happen if it were not perpendicular?

Show answer

The error vector must be perpendicular because that guarantees the shortest possible distance from the tip of b to the line. If the error were not perpendicular, you could "slide" the projection point along the line and find a position that is even closer to b. A perpendicular path between a point and a line always gives the minimum distance — this is the same reason that the shortest distance from your house to a straight road is the path that hits the road at a right angle.

Question 2: BASIC CALCULATION

Project the vector b = (6, 2) onto the line defined by a = (3, 1). Find proj_a(b) and the error vector e.

Show answer

First, compute a · b = (3)(6) + (1)(2) = 18 + 2 = 20. Then compute a · a = (3)(3) + (1)(1) = 9 + 1 = 10. The scalar coefficient is c = 20/10 = 2. So proj_a(b) = 2 × (3, 1) = (6, 2). The error is e = (6, 2) − (6, 2) = (0, 0). This means b already lies exactly on the line through a — it is its own projection! Verify: a · e = (3)(0) + (1)(0) = 0. ✓

Question 3: INTERMEDIATE

Project the vector b = (1, 5) onto the line defined by a = (2, −1). Compute the projection vector and verify that the error is perpendicular to a.

Show answer

Compute a · b = (2)(1) + (−1)(5) = 2 − 5 = −3. Compute a · a = (2)(2) + (−1)(−1) = 4 + 1 = 5. The scalar coefficient is c = −3/5. So proj_a(b) = (−3/5) × (2, −1) = (−6/5, 3/5) = (−1.2, 0.6). The error is e = (1, 5) − (−1.2, 0.6) = (2.2, 4.4). Verify: a · e = (2)(2.2) + (−1)(4.4) = 4.4 − 4.4 = 0. ✓ The negative scalar means the projection points in the opposite direction from a.

Question 4: APPLIED

A drone is at position b = (4, 7, 2) in 3D space. A power line runs along the direction a = (1, 0, 0) (the x-axis). Find the point on the power line closest to the drone and the distance from the drone to that point.

Show answer

Compute a · b = (1)(4) + (0)(7) + (0)(2) = 4. Compute a · a = (1)(1) + (0)(0) + (0)(0) = 1. So c = 4/1 = 4, and proj_a(b) = 4 × (1, 0, 0) = (4, 0, 0). The closest point on the power line is (4, 0, 0). The error vector is e = (4, 7, 2) − (4, 0, 0) = (0, 7, 2). The distance is ‖e‖ = √(0² + 7² + 2²) = √(0 + 49 + 4) = √53 ≈ 7.28 units. This is the shortest distance from the drone to the power line.

Question 5: CRITICAL THINKING

Suppose you project vector b onto vector a and get proj_a(b) = p. Now suppose you project p onto a again. What do you get? Explain why this result makes sense, and what property of projections it reveals.

Show answer

Projecting p onto a again gives you p itself. Since p already lies on the line through a (it was the result of projecting onto that line), projecting it a second time does not move it — it is already the closest point on the line to itself. Mathematically, if p = c × a, then proj_a(p) = (a · p)/(a · a) × a = (a · ca)/(a · a) × a = c(a · a)/(a · a) × a = c × a = p. This reveals that projection is an idempotent operation — applying it twice gives the same result as applying it once. Think of it like flattening something that is already flat: it stays flat.

SUMMARY

Summary

A projection finds the closest point on a line or subspace to a given vector. The formula for projecting b onto the line through a is proj_a(b) = (a · b / a · a) × a. The error vector e = b − proj_a(b) is always perpendicular to the target line, which is confirmed when a · e = 0.

This idea extends from lines to higher-dimensional subspaces like planes, where matrix operations replace simple dot products. Projections are the foundation of least-squares fitting, signal processing, and many applications in science and engineering. The core principle never changes: find the closest point, and make sure the leftover error is perpendicular to the target.