Best Approximation - Linear Algebra

In the inner product space $C[0,1]$ with inner product $\langle f, g \rangle = \int_0^1 f(x)g(x)dx$ , let $W = \text{span}\{1, x\}$ and $f(x) = x^2$ . The best approximation to $f$ in $W$ has the form $ax + b$ . What is the value of $a$ ?
Answer: $1$ When you encounter a "best approximation" problem in an inner product space, you're dealing with orthogonal projection onto a subspace. The best approximation to $f(x) = x^2$ in $W = \text{span}\{1, x\}$ is the orthogonal projection of $f$ onto $W$ . To find this projection, you need to use the Gram-Schmidt process or solve the normal equations. Since $\{1, x\}$ aren't orthogonal, let's first orthogonalize them. We have $u_1 = 1$ and $u_2 = x - \text{proj}_{u_1}(x)$ . Computing $\text{proj}_{u_1}(x) = \frac{\langle x, 1 \rangle}{\langle 1, 1 \rangle} \cdot 1 = \frac{\int_0^1 x dx}{\int_0^1 1 dx} = \frac{1/2}{1} = \frac{1}{2}$ . So $u_2 = x - \frac{1}{2}$ . Now the orthogonal projection of $x^2$ is: $\text{proj}_W(x^2) = \frac{\langle x^2, 1 \rangle}{\langle 1, 1 \rangle} \cdot 1 + \frac{\langle x^2, x - 1/2 \rangle}{\langle x - 1/2, x - 1/2 \rangle} \cdot (x - \frac{1}{2})$ After computing the integrals: $\langle x^2, 1 \rangle = 1/3$ , $\langle x^2, x - 1/2 \rangle = 1/12$ , and $\langle x - 1/2, x - 1/2 \rangle = 1/12$ . This gives us $\text{proj}_W(x^2) = \frac{1}{3} + 1 \cdot (x - \frac{1}{2}) = x - \frac{1}{6}$ . Therefore $a = 1$ (choice D). Choice A ( $\frac{1}{2}$ ) likely comes from incorrectly using the projection coefficient. Choice B ( $2$ ) might result from computational errors with the integrals. Choice C ( $\frac{3}{2}$ ) could arise from mishandling the orthogonalization process. Study tip: Master the orthogonal projection formula and practice computing inner products with polynomial functions—these integral calculations are crucial for success.
Let $W$ be the subspace of $\mathbb{R}^3$ spanned by the orthogonal basis ${\mathbf{u}_1 = \begin{pmatrix} 1 \\ -2 \\ 1 \end{pmatrix}, \mathbf{u}_2 = \begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix}}$ . For the vector $\mathbf{v} = \begin{pmatrix} 3 \\ 0 \\ 6 \end{pmatrix}$ , is the vector $\mathbf{w}_c = \begin{pmatrix} 2 \\ -1 \\ 2 \end{pmatrix}$ the best approximation of $\mathbf{v}$ in $W$ ?
Answer: No, because $\mathbf{v} - \mathbf{w}_c$ is not orthogonal to the basis vectors of $W$ . A vector $\mathbf{w}_c$ in $W$ is the best approximation of $\mathbf{v}$ if and only if the error vector $\mathbf{v} - \mathbf{w}_c$ is orthogonal to $W$ . To check this, we compute the dot product of the error vector with each basis vector of $W$ . First, compute the error vector: $\mathbf{z} = \mathbf{v} - \mathbf{w}_c = \begin{pmatrix} 3 \\ 0 \\ 6 \end{pmatrix} - \begin{pmatrix} 2 \\ -1 \\ 2 \end{pmatrix} = \begin{pmatrix} 1 \\ 1 \\ 4 \end{pmatrix}$ . Now, check for orthogonality: $\mathbf{z} \cdot \mathbf{u}_1 = \begin{pmatrix} 1 \\ 1 \\ 4 \end{pmatrix} \cdot \begin{pmatrix} 1 \\ -2 \\ 1 \end{pmatrix} = 1 - 2 + 4 = 3$ . Since the dot product is not zero, $\mathbf{z}$ is not orthogonal to $\mathbf{u}_1$ , and therefore $\mathbf{w}_c$ is not the best approximation. Choice A is a necessary but not sufficient condition. Choice B is true for many vectors in $W$ , not just the best one. Choice D is irrelevant.
Let $W$ be a subspace of $\mathbb{R}^n$ . If a vector $\mathbf{v}$ is an element of $W$ , what is the best approximation of $\mathbf{v}$ in $W$ ?
Answer: The vector $\mathbf{v}$ itself. The best approximation of $\mathbf{v}$ in $W$ is the vector $\mathbf{w} \in W$ that minimizes the distance $\|\mathbf{v} - \mathbf{w}\|$ . If $\mathbf{v}$ is already in $W$ , we can choose $\mathbf{w} = \mathbf{v}$ , which makes the distance $\|\mathbf{v} - \mathbf{v}\| = \|\mathbf{0}\| = 0$ . This is the minimum possible distance. Therefore, $\mathbf{v}$ is its own best approximation in $W$ .
In a signal processing application, a target signal is represented by the vector $\mathbf{s} = \begin{pmatrix} 10 \\ 5 \\ 15 \end{pmatrix}$ . This signal is to be approximated using a linear combination of two orthogonal basis signals, $\mathbf{b}_1 = \begin{pmatrix} 2 \\ -1 \\ -2 \end{pmatrix}$ and $\mathbf{b}_2 = \begin{pmatrix} 1 \\ 2 \\ 0 \end{pmatrix}$ . What is the resulting best-fit approximation of $\mathbf{s}$ ?
Answer: $\begin{pmatrix} 2/3 \\ 29/3 \\ 10/3 \end{pmatrix}$ The best-fit approximation is the orthogonal projection of $\mathbf{s}$ onto the subspace spanned by $\mathbf{b}_1$ and $\mathbf{b}_2$ . We calculate: $\hat{\mathbf{s}} = \frac{\mathbf{s} \cdot \mathbf{b}_1}{\mathbf{b}_1 \cdot \mathbf{b}_1}\mathbf{b}_1 + \frac{\mathbf{s} \cdot \mathbf{b}_2}{\mathbf{b}_2 \cdot \mathbf{b}_2}\mathbf{b}_2$ . First component: $\frac{10(2)+5(-1)+15(-2)}{2^2+(-1)^2+(-2)^2}\mathbf{b}_1 = \frac{20-5-30}{4+1+4}\mathbf{b}_1 = \frac{-15}{9}\mathbf{b}_1 = -\frac{5}{3}\mathbf{b}_1$ . Second component: $\frac{10(1)+5(2)+15(0)}{1^2+2^2+0^2}\mathbf{b}_2 = \frac{10+10}{1+4}\mathbf{b}_2 = \frac{20}{5}\mathbf{b}_2 = 4\mathbf{b}_2$ . Summing the components: $\hat{\mathbf{s}} = -\frac{5}{3}\begin{pmatrix} 2 \\ -1 \\ -2 \end{pmatrix} + 4\begin{pmatrix} 1 \\ 2 \\ 0 \end{pmatrix} = \begin{pmatrix} -10/3 \\ 5/3 \\ 10/3 \end{pmatrix} + \begin{pmatrix} 4 \\ 8 \\ 0 \end{pmatrix} = \begin{pmatrix} 2/3 \\ 29/3 \\ 10/3 \end{pmatrix}$ . Distractor A is only the second component of the projection. Distractor B results from a sign error in the first dot product. Distractor D is the error vector $\mathbf{s} - \hat{\mathbf{s}}$ .
Let $\mathbb{R}^n$ be equipped with the standard dot product. Let $W$ be a subspace of $\mathbb{R}^n$ and let $\mathbf{v}$ be a vector in $\mathbb{R}^n$ . A vector $\hat{\mathbf{w}}$ is the best approximation to $\mathbf{v}$ in $W$ . Which statement provides the unique defining property of $\hat{\mathbf{w}}$ ?
Answer: $\hat{\mathbf{w}}$ is the vector in $W$ such that $\mathbf{v} - \hat{\mathbf{w}}$ is orthogonal to every vector in $W$ . The best approximation of a vector $\mathbf{v}$ in a subspace $W$ is its orthogonal projection, $\hat{\mathbf{w}} = \mathrm{proj}_W(\mathbf{v})$ . The defining characteristic of the orthogonal projection is that the error vector, $\mathbf{v} - \hat{\mathbf{w}}$ , is orthogonal to the subspace $W$ . This means it must be orthogonal to every vector in $W$ . Choice B is insufficient; orthogonality to just $\hat{\mathbf{w}}$ is not enough. Choices A and D are generally false.
Let $A$ be an $m \times n$ matrix with $\text{rank}(A) = r < n$ , and consider the least squares problem $\min_{\mathbf{x}} ||A\mathbf{x} - \mathbf{b}||^2$ . If $\mathbf{x}_0$ is any solution to the normal equations $A^TA\mathbf{x} = A^T\mathbf{b}$ , which vector represents the best approximation to $\mathbf{b}$ in the column space of $A$ ?
Answer: $A\mathbf{x}_0$ , since this is the projection of $\mathbf{b}$ onto $\text{Col}(A)$ When you encounter least squares problems with rank-deficient matrices, remember that you're fundamentally seeking the best approximation within the column space, regardless of whether the solution is unique. The key insight is understanding what each component represents geometrically. Any solution $\mathbf{x}_0$ to the normal equations $A^TA\mathbf{x} = A^T\mathbf{b}$ minimizes $||A\mathbf{x} - \mathbf{b}||^2$ , even when the matrix is rank-deficient. The vector $A\mathbf{x}_0$ represents where this optimal $\mathbf{x}_0$ maps under the transformation $A$ . Since $A\mathbf{x}_0$ lies in the column space of $A$ and minimizes the distance to $\mathbf{b}$ , it is precisely the orthogonal projection of $\mathbf{b}$ onto $\text{Col}(A)$ . This projection is unique, even though $\mathbf{x}_0$ itself may not be. Choice A correctly identifies this projection property. Choice B confuses the solution vector $\mathbf{x}_0$ (which lives in $\mathbb{R}^n$ ) with the approximation to $\mathbf{b}$ (which lives in $\mathbb{R}^m$ ). Choice C misunderstands $A^T\mathbf{b}$ as the approximation, when this is actually part of the normal equations setup. Choice D identifies the residual vector $\mathbf{b} - A\mathbf{x}_0$ , which is orthogonal to the column space, not the best approximation within it. Study tip: In least squares problems, always distinguish between the solution vector $\mathbf{x}$ and the resulting approximation $A\mathbf{x}$ . The approximation is what matters for the original problem, and $A\mathbf{x}_0$ gives you the closest point in the column space.
Let $\mathbf{y} = \begin{pmatrix} 2 \\ 3 \\ 1 \end{pmatrix}$ and let $W$ be the subspace spanned by the orthogonal vectors $\mathbf{u}_1 = \begin{pmatrix} 1 \\ 0 \\ -1 \end{pmatrix}$ and $\mathbf{u}_2 = \begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix}$ . What is the shortest distance from $\mathbf{y}$ to $W$ ?
Answer: $\frac{\sqrt{6}}{2}$ The shortest distance from $\mathbf{y}$ to $W$ is the magnitude of the error vector $\|\mathbf{z}\| = \|\mathbf{y} - \hat{\mathbf{y}}\|$ , where $\hat{\mathbf{y}} = \mathrm{proj}_W(\mathbf{y})$ . First, find $\hat{\mathbf{y}} = \frac{\mathbf{y} \cdot \mathbf{u}_1}{\mathbf{u}_1 \cdot \mathbf{u}_1}\mathbf{u}_1 + \frac{\mathbf{y} \cdot \mathbf{u}_2}{\mathbf{u}_2 \cdot \mathbf{u}_2}\mathbf{u}_2$ . $\frac{2-1}{1+1}\mathbf{u}_1 + \frac{2+3+1}{1+1+1}\mathbf{u}_2 = \frac{1}{2}\mathbf{u}_1 + \frac{6}{3}\mathbf{u}_2 = \frac{1}{2}\mathbf{u}_1 + 2\mathbf{u}_2 = \begin{pmatrix} 1/2 \\ 0 \\ -1/2 \end{pmatrix} + \begin{pmatrix} 2 \\ 2 \\ 2 \end{pmatrix} = \begin{pmatrix} 5/2 \\ 2 \\ 3/2 \end{pmatrix}$ . Next, find the error vector $\mathbf{z} = \mathbf{y} - \hat{\mathbf{y}} = \begin{pmatrix} 2 \\ 3 \\ 1 \end{pmatrix} - \begin{pmatrix} 5/2 \\ 2 \\ 3/2 \end{pmatrix} = \begin{pmatrix} -1/2 \\ 1 \\ -1/2 \end{pmatrix}$ . Finally, find the magnitude: $\|\mathbf{z}\| = \sqrt{(-1/2)^2 + 1^2 + (-1/2)^2} = \sqrt{1/4 + 1 + 1/4} = \sqrt{3/2} = \frac{\sqrt{3}}{\sqrt{2}} = \frac{\sqrt{6}}{2}$ . Distractor B is the norm of the projection, $\|\hat{\mathbf{y}}\|$ . Distractor C is the norm of the original vector, $\|\mathbf{y}\|$ . Distractor D comes from a common calculation error.
Let $W$ be a subspace of $\mathbb{R}^n$ , and let $\mathbf{v}$ be a vector in $\mathbb{R}^n$ . Let $\hat{\mathbf{w}} = \mathrm{proj}_W(\mathbf{v})$ be the best approximation of $\mathbf{v}$ in $W$ . What is the best approximation of the vector $\mathbf{z} = \mathbf{v} - \hat{\mathbf{w}}$ in the same subspace $W$ ?
Answer: The zero vector, $\mathbf{0}$ . By the Orthogonal Decomposition Theorem, the vector $\mathbf{z} = \mathbf{v} - \hat{\mathbf{w}}$ is the component of $\mathbf{v}$ that is orthogonal to the subspace $W$ . This means $\mathbf{z}$ is an element of the orthogonal complement, $W^\perp$ . The best approximation of any vector in $W^\perp$ by vectors in $W$ is the zero vector, because $\mathbf{z}$ is already orthogonal to every vector in $W$ . Thus, $\mathrm{proj}_W(\mathbf{z}) = \mathbf{0}$ .
In $\mathbb{R}^3$ , let $L$ be the line through the origin with direction vector $\mathbf{d} = (2, -1, 2)$ . If $\mathbf{p} = (6, 3, 0)$ and $\mathbf{q}$ is the best approximation to $\mathbf{p}$ on $L$ , what is $|\mathbf{p} - \mathbf{q}|^2 + |\mathbf{q}|^2$ ?
Answer: $45$ This question tests orthogonal projection and the Pythagorean theorem in vector spaces. When you see "best approximation" of a point onto a line or subspace, you're looking for the orthogonal projection, which minimizes the distance. To find the projection $\mathbf{q}$ of $\mathbf{p} = (6, 3, 0)$ onto line $L$ with direction $\mathbf{d} = (2, -1, 2)$ , use the formula: $\mathbf{q} = \frac{\mathbf{p} \cdot \mathbf{d}}{\mathbf{d} \cdot \mathbf{d}} \mathbf{d}$ First, calculate the dot products: - $\mathbf{p} \cdot \mathbf{d} = 6(2) + 3(-1) + 0(2) = 12 - 3 = 9$ - $\mathbf{d} \cdot \mathbf{d} = 4 + 1 + 4 = 9$ So $\mathbf{q} = \frac{9}{9}(2, -1, 2) = (2, -1, 2)$ The error vector is $\mathbf{p} - \mathbf{q} = (6, 3, 0) - (2, -1, 2) = (4, 4, -2)$ Now calculate: $|\mathbf{p} - \mathbf{q}|^2 = 16 + 16 + 4 = 36$ and $|\mathbf{q}|^2 = 4 + 1 + 4 = 9$ Therefore, $|\mathbf{p} - \mathbf{q}|^2 + |\mathbf{q}|^2 = 36 + 9 = 45$ , which is answer A. The wrong answers likely come from calculation errors: B (36) uses only $|\mathbf{p} - \mathbf{q}|^2$ , forgetting to add $|\mathbf{q}|^2$ . C (49) might result from using $|\mathbf{p}|^2 = 45$ plus a small error. D (81) could come from incorrectly using $|\mathbf{p}|^2 + |\mathbf{q}|^2$ . Remember: orthogonal projection problems often involve the Pythagorean theorem. The projection and error vectors are perpendicular, so $|\mathbf{p}|^2 = |\mathbf{q}|^2 + |\mathbf{p} - \mathbf{q}|^2$ .
Consider the matrix $A = \begin{pmatrix} 1 & 2 \\ 2 & 4 \\ 1 & 2 \end{pmatrix}$ and vector $\mathbf{b} = \begin{pmatrix} 3 \\ 7 \\ 2 \end{pmatrix}$ . The least squares solution $\hat{\mathbf{x}}$ to $A\mathbf{x} = \mathbf{b}$ minimizes $||A\mathbf{x} - \mathbf{b}||^2$ . What is the minimum value of this expression?
Answer: $\frac{5}{2}$ Since $A$ has rank 1 with $\text{Col}(A) = \text{span}\{(1,2,1)^T\}$ , the best approximation to $\mathbf{b}$ in $\text{Col}(A)$ is $\text{proj}_{\text{Col}(A)}(\mathbf{b}) = \frac{\mathbf{b} \cdot (1,2,1)}{||(1,2,1)||^2}(1,2,1) = \frac{3 + 14 + 2}{6}(1,2,1) = \frac{19}{6}(1,2,1) = (\frac{19}{6}, \frac{19}{3}, \frac{19}{6})$ . The minimum residual is $||\mathbf{b} - \text{proj}_{\text{Col}(A)}(\mathbf{b})||^2 = ||(3,7,2) - (\frac{19}{6}, \frac{19}{3}, \frac{19}{6})||^2 = ||(-\frac{1}{6}, \frac{2}{3}, -\frac{7}{6})||^2 = \frac{1}{36} + \frac{4}{9} + \frac{49}{36} = \frac{1 + 16 + 49}{36} = \frac{66}{36} = \frac{11}{6}$ . Wait, let me recalculate: $\frac{1}{36} + \frac{16}{36} + \frac{49}{36} = \frac{66}{36} = \frac{11}{6}$ . Actually, $\frac{4}{9} = \frac{16}{36}$ , so the total is $\frac{1 + 16 + 49}{36} = \frac{66}{36} = \frac{11}{6}$ . Hmm, this doesn't match the choices. Let me recalculate the projection: $\mathbf{b} \cdot (1,2,1) = 3 + 14 + 2 = 19$ , $||(1,2,1)||^2 = 1 + 4 + 1 = 6$ . So $\text{proj} = \frac{19}{6}(1,2,1)$ . The error is $(3,7,2) - (\frac{19}{6}, \frac{38}{6}, \frac{19}{6}) = (\frac{18-19}{6}, \frac{42-38}{6}, \frac{12-19}{6}) = (-\frac{1}{6}, \frac{4}{6}, -\frac{7}{6}) = (-\frac{1}{6}, \frac{2}{3}, -\frac{7}{6})$ . Then $||\text{error}||^2 = \frac{1}{36} + \frac{4}{9} + \frac{49}{36} = \frac{1 + 16 + 49}{36} = \frac{90}{36} = \frac{5}{2}$ . Choices A, B, D represent computational errors in the projection or residual calculation.
Let $W$ be the subspace of $\mathbb{R}^3$ spanned by the orthogonal vectors $\mathbf{u}_1 = \begin{pmatrix} 1 \\ 1 \\ 0 \end{pmatrix}$ and $\mathbf{u}_2 = \begin{pmatrix} 1 \\ -1 \\ 2 \end{pmatrix}$ . Find the best approximation of $\mathbf{v} = \begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix}$ in $W$ .
Answer: $\begin{pmatrix} 7/3 \\ 2/3 \\ 5/3 \end{pmatrix}$ Since the basis for $W$ is orthogonal, the best approximation (the projection) is the sum of the projections onto each basis vector: $\hat{\mathbf{v}} = \frac{\mathbf{v} \cdot \mathbf{u}_1}{\mathbf{u}_1 \cdot \mathbf{u}_1}\mathbf{u}_1 + \frac{\mathbf{v} \cdot \mathbf{u}_2}{\mathbf{u}_2 \cdot \mathbf{u}_2}\mathbf{u}_2$ . Projection onto $\mathbf{u}_1$ : $\frac{1(1)+2(1)+3(0)}{1^2+1^2+0^2}\mathbf{u}_1 = \frac{3}{2}\mathbf{u}_1 = \begin{pmatrix} 3/2 \\ 3/2 \\ 0 \end{pmatrix}$ . Projection onto $\mathbf{u}_2$ : $\frac{1(1)+2(-1)+3(2)}{1^2+(-1)^2+2^2}\mathbf{u}_2 = \frac{1-2+6}{1+1+4}\mathbf{u}_2 = \frac{5}{6}\mathbf{u}_2 = \begin{pmatrix} 5/6 \\ -5/6 \\ 10/6 \end{pmatrix} = \begin{pmatrix} 5/6 \\ -5/6 \\ 5/3 \end{pmatrix}$ . Summing the two projections: $\begin{pmatrix} 3/2 \\ 3/2 \\ 0 \end{pmatrix} + \begin{pmatrix} 5/6 \\ -5/6 \\ 5/3 \end{pmatrix} = \begin{pmatrix} 9/6+5/6 \\ 9/6-5/6 \\ 5/3 \end{pmatrix} = \begin{pmatrix} 14/6 \\ 4/6 \\ 5/3 \end{pmatrix} = \begin{pmatrix} 7/3 \\ 2/3 \\ 5/3 \end{pmatrix}$ . Distractors A and B are the individual projections. Distractor C is the error vector $\mathbf{v} - \hat{\mathbf{v}}$ .
The component of vector $\mathbf{v} = \begin{pmatrix} 3 \\ -1 \\ 5 \end{pmatrix}$ orthogonal to the subspace $W$ spanned by $\mathbf{u} = \begin{pmatrix} 1 \\ 2 \\ 2 \end{pmatrix}$ is:
Answer: $\begin{pmatrix} 16/9 \\ -31/9 \\ 23/9 \end{pmatrix}$ The component of $\mathbf{v}$ orthogonal to $W$ is the error vector $\mathbf{z} = \mathbf{v} - \hat{\mathbf{w}}$ , where $\hat{\mathbf{w}} = \mathrm{proj}_W(\mathbf{v})$ . First, find the projection of $\mathbf{v}$ onto the line spanned by $\mathbf{u}$ : $\hat{\mathbf{w}} = \frac{\mathbf{v} \cdot \mathbf{u}}{\mathbf{u} \cdot \mathbf{u}}\mathbf{u} = \frac{3(1)+(-1)(2)+5(2)}{1^2+2^2+2^2}\mathbf{u} = \frac{3-2+10}{1+4+4}\mathbf{u} = \frac{11}{9}\mathbf{u} = \begin{pmatrix} 11/9 \\ 22/9 \\ 22/9 \end{pmatrix}$ . Next, subtract this projection from $\mathbf{v}$ : $\mathbf{z} = \mathbf{v} - \hat{\mathbf{w}} = \begin{pmatrix} 3 \\ -1 \\ 5 \end{pmatrix} - \begin{pmatrix} 11/9 \\ 22/9 \\ 22/9 \end{pmatrix} = \begin{pmatrix} 27/9 - 11/9 \\ -9/9 - 22/9 \\ 45/9 - 22/9 \end{pmatrix} = \begin{pmatrix} 16/9 \\ -31/9 \\ 23/9 \end{pmatrix}$ . Distractor A is the projection itself. Distractor B comes from a sign error in the dot product calculation. Distractor D is the original vector.
Let $S = \{\mathbf{u}_1, \mathbf{u}_2\}$ be an orthonormal set in an inner product space $V$ , and let $W = \text{span}(S)$ . If $\mathbf{v} \in V$ and $\langle \mathbf{v}, \mathbf{u}_1 \rangle = 4$ and $\langle \mathbf{v}, \mathbf{u}_2 \rangle = -3$ , what is $||\mathbf{v} - \text{proj}_W(\mathbf{v})||^2$ ?
Answer: $||\mathbf{v}||^2 - 25$ Since $S$ is orthonormal, $\text{proj}_W(\mathbf{v}) = \langle \mathbf{v}, \mathbf{u}_1 \rangle \mathbf{u}_1 + \langle \mathbf{v}, \mathbf{u}_2 \rangle \mathbf{u}_2 = 4\mathbf{u}_1 - 3\mathbf{u}_2$ . By the Pythagorean theorem for orthogonal decomposition, $||\mathbf{v}||^2 = ||\text{proj}_W(\mathbf{v})||^2 + ||\mathbf{v} - \text{proj}_W(\mathbf{v})||^2$ . Since $||\text{proj}_W(\mathbf{v})||^2 = ||4\mathbf{u}_1 - 3\mathbf{u}_2||^2 = 16||\mathbf{u}_1||^2 + 9||\mathbf{u}_2||^2 = 16 + 9 = 25$ , we have $||\mathbf{v} - \text{proj}_W(\mathbf{v})||^2 = ||\mathbf{v}||^2 - 25$ . Choice A incorrectly subtracts $|4 - 3| = 7$ . Choice C uses $|4^2 - 3^2| = 7$ then makes an arithmetic error. Choice D adds instead of subtracts the projection norm squared.
Find the best approximation of the vector $\mathbf{v} = \begin{pmatrix} 7 \\ 6 \end{pmatrix}$ by a vector on the line spanned by $\mathbf{u} = \begin{pmatrix} 4 \\ 2 \end{pmatrix}$ .
Answer: $\begin{pmatrix} 8 \\ 4 \end{pmatrix}$ The best approximation is the orthogonal projection of $\mathbf{v}$ onto the line spanned by $\mathbf{u}$ . The formula is $\hat{\mathbf{v}} = \frac{\mathbf{v} \cdot \mathbf{u}}{\mathbf{u} \cdot \mathbf{u}}\mathbf{u}$ . First, calculate the dot products: $\mathbf{v} \cdot \mathbf{u} = (7)(4) + (6)(2) = 28 + 12 = 40$ . $\mathbf{u} \cdot \mathbf{u} = (4)(4) + (2)(2) = 16 + 4 = 20$ . The scalar coefficient is $\frac{40}{20} = 2$ . Therefore, the projection is $2\mathbf{u} = 2\begin{pmatrix} 4 \\ 2 \end{pmatrix} = \begin{pmatrix} 8 \\ 4 \end{pmatrix}$ . Distractor B is the error vector $\mathbf{v} - \hat{\mathbf{v}}$ . Distractors C and D represent naive geometric attempts to scale $\mathbf{u}$ to match one component of $\mathbf{v}$ .
Let $W$ be a subspace with an orthonormal basis ${\mathbf{q}_1, \mathbf{q}_2}$ , where $\mathbf{q}_1 = \frac{1}{\sqrt{2}}\begin{pmatrix} 1 \\ 1 \\ 0 \end{pmatrix}$ and $\mathbf{q}_2 = \frac{1}{\sqrt{6}}\begin{pmatrix} 1 \\ -1 \\ 2 \end{pmatrix}$ . Find the best approximation of $\mathbf{v} = \begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix}$ in $W$ .
Answer: $\begin{pmatrix} 7/3 \\ 2/3 \\ 5/3 \end{pmatrix}$ For an orthonormal basis, the projection formula simplifies to $\hat{\mathbf{v}} = (\mathbf{v} \cdot \mathbf{q}_1)\mathbf{q}_1 + (\mathbf{v} \cdot \mathbf{q}_2)\mathbf{q}_2$ . Calculate the dot products: $\mathbf{v} \cdot \mathbf{q}_1 = 1(\frac{1}{\sqrt{2}}) + 2(\frac{1}{\sqrt{2}}) + 3(0) = \frac{3}{\sqrt{2}}$ . $\mathbf{v} \cdot \mathbf{q}_2 = 1(\frac{1}{\sqrt{6}}) + 2(\frac{-1}{\sqrt{6}}) + 3(\frac{2}{\sqrt{6}}) = \frac{1-2+6}{\sqrt{6}} = \frac{5}{\sqrt{6}}$ . Now substitute these scalars into the formula: $\hat{\mathbf{v}} = (\frac{3}{\sqrt{2}})\mathbf{q}_1 + (\frac{5}{\sqrt{6}})\mathbf{q}_2 = (\frac{3}{\sqrt{2}})\frac{1}{\sqrt{2}}\begin{pmatrix} 1 \\ 1 \\ 0 \end{pmatrix} + (\frac{5}{\sqrt{6}})\frac{1}{\sqrt{6}}\begin{pmatrix} 1 \\ -1 \\ 2 \end{pmatrix}$ . $\hat{\mathbf{v}} = \frac{3}{2}\begin{pmatrix} 1 \\ 1 \\ 0 \end{pmatrix} + \frac{5}{6}\begin{pmatrix} 1 \\ -1 \\ 2 \end{pmatrix} = \begin{pmatrix} 3/2 \\ 3/2 \\ 0 \end{pmatrix} + \begin{pmatrix} 5/6 \\ -5/6 \\ 10/6 \end{pmatrix} = \begin{pmatrix} 7/3 \\ 2/3 \\ 5/3 \end{pmatrix}$ . Distractor A results from a sign error in the second dot product. Distractor B is only the projection onto $\mathbf{q}_1$ . Distractor C is the error vector.
Let $W$ be the plane in $\mathbb{R}^3$ spanned by $\mathbf{x}_1 = \begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix}$ and $\mathbf{x}_2 = \begin{pmatrix} 1 \\ 0 \\ 2 \end{pmatrix}$ . Which vector in $W$ is closest to $\mathbf{v} = \begin{pmatrix} 0 \\ 3 \\ 3 \end{pmatrix}$ ?
Answer: $\begin{pmatrix} 2 \\ 2 \\ 2 \end{pmatrix}$ The basis vectors $\mathbf{x}_1$ and $\mathbf{x}_2$ are not orthogonal, since $\mathbf{x}_1 \cdot \mathbf{x}_2 = 1+0+2 = 3 \neq 0$ . We must first find an orthogonal basis for $W$ using the Gram-Schmidt process. Let $\mathbf{u}_1 = \mathbf{x}_1 = \begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix}$ . Let $\mathbf{u}_2 = \mathbf{x}_2 - \frac{\mathbf{x}_2 \cdot \mathbf{u}_1}{\mathbf{u}_1 \cdot \mathbf{u}_1}\mathbf{u}_1 = \begin{pmatrix} 1 \\ 0 \\ 2 \end{pmatrix} - \frac{3}{3}\begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix} = \begin{pmatrix} 1 \\ 0 \\ 2 \end{pmatrix} - \begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix} = \begin{pmatrix} 0 \\ -1 \\ 1 \end{pmatrix}$ . Now, project $\mathbf{v}$ onto the orthogonal basis ${\mathbf{u}_1, \mathbf{u}_2}$ : $\hat{\mathbf{v}} = \frac{\mathbf{v} \cdot \mathbf{u}_1}{\mathbf{u}_1 \cdot \mathbf{u}_1}\mathbf{u}_1 + \frac{\mathbf{v} \cdot \mathbf{u}_2}{\mathbf{u}_2 \cdot \mathbf{u}_2}\mathbf{u}_2 = \frac{0+3+3}{3}\mathbf{u}_1 + \frac{0-3+3}{2}\mathbf{u}_2 = 2\mathbf{u}_1 + 0\mathbf{u}_2 = 2\begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix} = \begin{pmatrix} 2 \\ 2 \\ 2 \end{pmatrix}$ . Distractor A is the incorrect result from using the non-orthogonal basis directly in the projection formula. Distractor D is the error vector $\mathbf{v} - \hat{\mathbf{v}}$ .
Let $\mathbf{v}_1 = (1, 2, 1)$ , $\mathbf{v}_2 = (0, 1, 2)$ , and $\mathbf{v}_3 = (2, 3, 0)$ in $\mathbb{R}^3$ . If $W = \text{span}\{\mathbf{v}_1, \mathbf{v}_2\}$ and $\mathbf{b} = (3, 4, 5)$ , which statement about the best approximation $\hat{\mathbf{b}}$ to $\mathbf{b}$ in $W$ is correct?
Answer: $\mathbf{v}_3 \cdot (\mathbf{b} - \hat{\mathbf{b}}) \neq 0$ since $\mathbf{v}_3 \notin W^\perp$ When you encounter questions about orthogonal projections and best approximations, the fundamental principle is that the error vector $\mathbf{b} - \hat{\mathbf{b}}$ must be orthogonal to the subspace $W$ . This means the error is perpendicular to every vector in $W$ , including the spanning vectors. The correct answer is D. To see why, we need to determine if $\mathbf{v}_3$ is in $W^\perp$ (the orthogonal complement of $W$ ). For $\mathbf{v}_3$ to be in $W^\perp$ , it must be orthogonal to both spanning vectors of $W$ . Let's check: $\mathbf{v}_1 \cdot \mathbf{v}_3 = (1)(2) + (2)(3) + (1)(0) = 8 \neq 0$ . Since $\mathbf{v}_3$ is not orthogonal to $\mathbf{v}_1$ , we have $\mathbf{v}_3 \notin W^\perp$ . Therefore, $\mathbf{v}_3 \cdot (\mathbf{b} - \hat{\mathbf{b}}) \neq 0$ . A is wrong because $\mathbf{v}_3 \notin W^\perp$ , as we just showed. B is incorrect because there's no "symmetry" principle that would make the dot products with $\mathbf{v}_1$ and $\mathbf{v}_2$ equal. C misunderstands the orthogonality condition: since $\mathbf{v}_1 \in W$ , we must have $\mathbf{v}_1 \cdot (\mathbf{b} - \hat{\mathbf{b}}) = 0$ for any orthogonal projection, regardless of whether the projection is "exact." Key takeaway: Always check if a vector belongs to $W^\perp$ by testing orthogonality with all spanning vectors of $W$ . Only vectors in $W^\perp$ will have zero dot product with the error vector.
What point in the plane spanned by the orthogonal vectors $\mathbf{u}_1 = \begin{pmatrix} 1 \\ 0 \\ -2 \end{pmatrix}$ and $\mathbf{u}_2 = \begin{pmatrix} 2 \\ 1 \\ 1 \end{pmatrix}$ is closest to the point $P(3, 3, 3)$ ?
Answer: $(17/5, 2, 16/5)$ This is equivalent to finding the best approximation of the vector $\mathbf{v} = \begin{pmatrix} 3 \\ 3 \\ 3 \end{pmatrix}$ in the subspace $W$ spanned by ${\mathbf{u}_1, \mathbf{u}_2}$ . The closest point is the endpoint of the projection vector $\hat{\mathbf{v}} = \mathrm{proj}_W(\mathbf{v})$ . $\hat{\mathbf{v}} = \frac{\mathbf{v} \cdot \mathbf{u}_1}{\mathbf{u}_1 \cdot \mathbf{u}_1}\mathbf{u}_1 + \frac{\mathbf{v} \cdot \mathbf{u}_2}{\mathbf{u}_2 \cdot \mathbf{u}_2}\mathbf{u}_2$ . Projection onto $\mathbf{u}_1$ : $\frac{3(1)+3(0)+3(-2)}{1^2+0^2+(-2)^2}\mathbf{u}_1 = \frac{-3}{5}\mathbf{u}_1 = \begin{pmatrix} -3/5 \\ 0 \\ 6/5 \end{pmatrix}$ . Projection onto $\mathbf{u}_2$ : $\frac{3(2)+3(1)+3(1)}{2^2+1^2+1^2}\mathbf{u}_2 = \frac{12}{6}\mathbf{u}_2 = 2\mathbf{u}_2 = \begin{pmatrix} 4 \\ 2 \\ 2 \end{pmatrix}$ . Summing them: $\hat{\mathbf{v}} = \begin{pmatrix} -3/5 \\ 0 \\ 6/5 \end{pmatrix} + \begin{pmatrix} 4 \\ 2 \\ 2 \end{pmatrix} = \begin{pmatrix} -3/5+20/5 \\ 2 \\ 6/5+10/5 \end{pmatrix} = \begin{pmatrix} 17/5 \\ 2 \\ 16/5 \end{pmatrix}$ . The corresponding point is $(17/5, 2, 16/5)$ . Distractors A and C are the individual projection components. Distractor D is the error vector.

Start this quiz