QR Factorization - Linear Algebra

1. A $3 \times 2$ matrix $A$ has QR factorization $A = QR$ where $Q$ is $3 \times 2$ and $R$ is $2 \times 2$. If $||A\mathbf{x}||^2 = 17$ for some vector $\mathbf{x} \in \mathbb{R}^2$, what is the value of $||R\mathbf{x}||^2$?
   Answer: $17$ Since $A = QR$ and $Q$ has orthonormal columns, $Q^TQ = I_2$. Therefore, $||A\mathbf{x}||^2 = ||QR\mathbf{x}||^2 = (QR\mathbf{x})^T(QR\mathbf{x}) = \mathbf{x}^TR^TQ^TQR\mathbf{x} = \mathbf{x}^TR^TR\mathbf{x} = ||R\mathbf{x}||^2 = 17$. Choice B incorrectly takes a square root. Choice C fails to recognize that orthonormality of $Q$ makes the result independent of the specific form of $Q$. Choice D incorrectly suggests the relationship depends on the specific vector $\mathbf{x}$ rather than being a general property.
2. Let $A$ be a $4 \times 3$ matrix with QR factorization $A = QR$, where $Q$ is $4 \times 3$ with orthonormal columns and $R$ is $3 \times 3$ upper triangular. If the second column of $A$ is orthogonal to the first column of $A$, which of the following must be true about the matrix $R$?
   Answer: $R_{12} = 0$ only In the Gram-Schmidt process, $R_{ij} = \langle \mathbf{a}_j, \mathbf{q}_i \rangle$ for $i \leq j$. Since $\mathbf{a}_2 \perp \mathbf{a}_1$, we have $\langle \mathbf{a}_2, \mathbf{q}_1 \rangle = \frac{\langle \mathbf{a}_2, \mathbf{a}_1 \rangle}{||\mathbf{a}_1||} = 0$, so $R_{12} = 0$. Since $R$ is upper triangular, $R_{21} = 0$ by definition, not due to orthogonality. Choice A incorrectly suggests both are consequences of orthogonality. Choice C misses the key implication. Choice D makes an unrelated claim about diagonal entries.
3. A $5 \times 3$ matrix $A$ has rank 2. When computing the QR factorization using Gram-Schmidt orthogonalization, what happens during the process when reaching the third column?
   Answer: The third column becomes a zero vector after orthogonalization, requiring special handling to complete the factorization When you encounter QR factorization problems involving rank-deficient matrices, focus on what happens during the Gram-Schmidt orthogonalization process when columns are linearly dependent. In Gram-Schmidt, each new column is made orthogonal to all previous columns by subtracting their projections. For a $5 \times 3$ matrix with rank 2, the first two columns are linearly independent and will orthogonalize normally. However, the third column must be a linear combination of the first two (since the column space is only 2-dimensional). When you apply Gram-Schmidt to the third column, you subtract its projections onto the first two orthogonal vectors. Since the third column lies entirely in the span of the first two, these projections account for the entire vector - leaving nothing behind. The result is the zero vector, which cannot be normalized to create a unit vector for the $Q$ matrix. This is why answer C is correct: special handling is needed to complete the factorization, typically by adding an orthonormal vector to span the full space. Answer A is wrong because QR factorization can handle rank-deficient matrices with modifications. Answer B is incorrect because if the third column were already orthogonal to the first two, the matrix would have rank 3, not 2. Answer D mischaracterizes the process - the algorithm doesn't arbitrarily replace columns; it follows the mathematical consequence of linear dependence. Study tip: Remember that Gram-Schmidt reveals linear dependence by producing zero vectors - this is your signal that special techniques are needed for completion.
4. Let $A$ be an $m \times n$ matrix with $n$ linearly independent columns. Its QR factorization is $A=QR$, where $Q$ is an $m \times n$ matrix and $R$ is an $n \times n$ matrix. Which statement most accurately describes the columns of the matrix $Q$?
   Answer: They form an orthonormal basis for the column space of $A$. The QR factorization is constructed by applying the Gram-Schmidt process to the columns of $A$. This process explicitly creates a set of orthonormal vectors that span the same space as the original set of vectors. Therefore, the columns of $Q$ are an orthonormal set of vectors that form a basis for the column space of $A$. Distractor C is incorrect because the columns of Q are normalized to have unit length, making them orthonormal, not just orthogonal. The other distractors refer to incorrect fundamental subspaces.
5. Given the QR factorization of a matrix $A$, a linear system $A\mathbf{x} = \mathbf{b}$ can be solved more easily. The first step is to transform the system into an equivalent upper triangular system. What is this equivalent system?
   Answer: $R\mathbf{x} = Q^T\mathbf{b}$ Starting with $A\mathbf{x} = \mathbf{b}$, we substitute $A=QR$ to get $QR\mathbf{x} = \mathbf{b}$. Since the columns of $Q$ are orthonormal, $Q^TQ=I$. We can multiply both sides of the equation on the left by $Q^T$: $Q^T(QR\mathbf{x}) = Q^T\mathbf{b}$, which simplifies to $(Q^TQ)R\mathbf{x} = Q^T\mathbf{b}$, and then $IR\mathbf{x} = Q^T\mathbf{b}$. This gives the equivalent system $R\mathbf{x} = Q^T\mathbf{b}$. This system is easy to solve using back substitution because $R$ is upper triangular.
6. Let $A$ be a $4 \times 2$ matrix with linearly independent columns, and let $A = QR$ be its QR factorization. If $\mathbf{b}$ is a vector in $\mathbb{R}^4$ such that $Q^T\mathbf{b} = \begin{pmatrix} 5 \\ -2 \end{pmatrix}$, what is the solution to the least squares problem $\min_{\mathbf{x}} ||A\mathbf{x} - \mathbf{b}||^2$?
   Answer: $\mathbf{x} = R^{-1}\begin{pmatrix} 5 \\ -2 \end{pmatrix}$ When you encounter QR factorization in least squares problems, you're dealing with an elegant computational shortcut that avoids the numerical instability of the normal equations. The key insight is that for the least squares problem $\min_{\mathbf{x}} ||A\mathbf{x} - \mathbf{b}||^2$, the normal equations give us $A^T A \mathbf{x} = A^T \mathbf{b}$. When $A = QR$, we can substitute: $(QR)^T(QR)\mathbf{x} = (QR)^T\mathbf{b}$, which simplifies to $R^TR\mathbf{x} = R^TQ^T\mathbf{b}$. Since $R$ is upper triangular and invertible (because $A$ has linearly independent columns), we can multiply both sides by $(R^T)^{-1}$ to get $R\mathbf{x} = Q^T\mathbf{b}$. Therefore, $\mathbf{x} = R^{-1}Q^T\mathbf{b}$. Since we're given that $Q^T\mathbf{b} = \begin{pmatrix} 5 \\ -2 \end{pmatrix}$, the solution is $\mathbf{x} = R^{-1}\begin{pmatrix} 5 \\ -2 \end{pmatrix}$, making C correct. Option A represents the traditional normal equations approach but incorrectly applies $(A^TA)^{-1}$ to $Q^T\mathbf{b}$ instead of $A^T\mathbf{b}$. Option B gives you a vector in $\mathbb{R}^4$, but $\mathbf{x}$ must be in $\mathbb{R}^2$ to match $A$'s column space. Option D ignores the matrix $R$ entirely, which would only work if $R$ were the identity matrix. Remember: QR factorization transforms least squares into the simpler system $R\mathbf{x} = Q^T\mathbf{b}$. This is computationally superior to forming $A^TA$ explicitly.
7. Consider the matrix $A = \begin{pmatrix} 1 & 1 & 0 \\ 1 & 0 & 1 \\ 0 & 1 & 1 \end{pmatrix}$. If $A = QR$ is the QR factorization where $Q$ has orthonormal columns, what is the entry in position $(2,1)$ of the matrix $Q$?
   Answer: $\frac{1}{\sqrt{2}}$ To find Q, we apply Gram-Schmidt to the columns of A. Let the columns be $\mathbf{a}_1 = (1,1,0)^T$, $\mathbf{a}_2 = (1,0,1)^T$, $\mathbf{a}_3 = (0,1,1)^T$. First, $\mathbf{u}_1 = \mathbf{a}_1 = (1,1,0)^T$ with $||\mathbf{u}_1|| = \sqrt{2}$, so $\mathbf{q}_1 = \frac{1}{\sqrt{2}}(1,1,0)^T$. The $(2,1)$ entry of Q is the second component of $\mathbf{q}_1$, which is $\frac{1}{\sqrt{2}}$. Choice B gives the incorrect normalization factor. Choice C involves an erroneous calculation mixing norms. Choice D uses the wrong denominator from incomplete normalization.
8. Let $A = \begin{pmatrix} 1 & 1 \\ 1 & 3 \\ 1 & 1 \\ 1 & 3 \end{pmatrix}$. In the QR factorization $A=QR$, the first column of $Q$ is $\mathbf{q}_1 = [\frac{1}{2}, \frac{1}{2}, \frac{1}{2}, \frac{1}{2}]^T$. What is the value of the entry $r_{12}$ in the matrix $R$?
   Answer: $4$ The entries of the matrix $R$ in the QR factorization are given by $r_{ij} = \mathbf{q}_i^T \mathbf{a}_j$. To find $r_{12}$, we compute the dot product of the first column of $Q$ and the second column of $A$. Given $\mathbf{q}_1 = [\frac{1}{2}, \frac{1}{2}, \frac{1}{2}, \frac{1}{2}]^T$ and $\mathbf{a}_2 = [1, 3, 1, 3]^T$, the calculation is: $r_{12} = \mathbf{q}_1^T \mathbf{a}_2 = (\frac{1}{2})(1) + (\frac{1}{2})(3) + (\frac{1}{2})(1) + (\frac{1}{2})(3) = \frac{1}{2} + \frac{3}{2} + \frac{1}{2} + \frac{3}{2} = \frac{8}{2} = 4$.
9. Let $A$ be an invertible $n \times n$ matrix with QR factorization $A=QR$, where $Q$ is an orthogonal matrix and $R$ is an upper triangular matrix with positive diagonal entries. Which expression correctly gives the determinant of $A$?
   Answer: $\det(A) = \pm \det(R)$ Using the property that the determinant of a product of matrices is the product of their determinants, we have $\det(A) = \det(QR) = \det(Q)\det(R)$. Since $Q$ is an orthogonal matrix, its columns are orthonormal. This means $Q^TQ = I$. Taking the determinant gives $\det(Q^T)\det(Q) = \det(I)$, which is $(\det(Q))^2 = 1$. Therefore, $\det(Q)$ must be either $1$ or $-1$. Substituting this into the equation for $\det(A)$ gives $\det(A) = (\pm 1)\det(R) = \pm \det(R)$.
10. Consider a $3 \times 3$ matrix $A$ whose columns have norms $||\mathbf{a}_1|| = 2$, $||\mathbf{a}_2|| = 3$, and $||\mathbf{a}_3|| = 1$. After applying Gram-Schmidt orthogonalization, the resulting orthogonal vectors have norms $||\mathbf{u}_1|| = 2$, $||\mathbf{u}_2|| = \sqrt{5}$, and $||\mathbf{u}_3|| = \frac{2}{3}$. In the QR factorization $A = QR$, what is $R_{22}$?
    Answer: $\sqrt{5}$ In the QR factorization, the diagonal entries of $R$ are the norms of the orthogonalized vectors from Gram-Schmidt: $R_{ii} = ||\mathbf{u}_i||$. Therefore, $R_{22} = ||\mathbf{u}_2|| = \sqrt{5}$. Choice A incorrectly uses the original norm of $\mathbf{a}_2$. Choice C uses the norm of $\mathbf{u}_3$. Choice D incorrectly attempts to normalize $||\mathbf{u}_2||$ by the original norm of $\mathbf{a}_2$, which is not how the QR factorization works.
11. Let $A$ be an $m \times n$ matrix with linearly independent columns, and let $A=QR$ be its QR factorization. Which of the following statements is always true?
    Answer: The columns of $Q$ form an orthonormal basis for the column space of $A$. The QR factorization is constructed using the Gram-Schmidt process on the columns of $A$. This process generates an orthonormal set of vectors that spans the same subspace as the original columns. Therefore, the columns of $Q$ form an orthonormal basis for the column space of $A$. Distractor A is false because $Q$ is only square (and thus invertible) if $m=n$. Distractor B is false; $R$ is upper triangular, not necessarily symmetric. Distractor D is false because $\det(A) = \det(Q)\det(R)$, and $\det(Q)$ is not always 1 (it can be -1).
12. Let $\mathbf{a}_1 = [1, 1, 0]^T$ and $\mathbf{a}_2 = [1, 2, 2]^T$. When applying the Gram-Schmidt process to the set of vectors $\{\mathbf{a}_1, \mathbf{a}_2\}$, what is the resulting second orthogonal vector $\mathbf{v}_2$?
    Answer: $[-\frac{1}{2}, \frac{1}{2}, 2]^T$ The Gram-Schmidt process sets $\mathbf{v}_1 = \mathbf{a}_1 = [1, 1, 0]^T$. The second orthogonal vector $\mathbf{v}_2$ is found by subtracting the projection of $\mathbf{a}_2$ onto $\mathbf{v}_1$ from $\mathbf{a}_2$. The projection is $\mathrm{proj}_{\mathbf{v}_1}(\mathbf{a}_2) = \frac{\mathbf{a}_2 \cdot \mathbf{v}_1}{\|\mathbf{v}_1\|^2}\mathbf{v}_1 = \frac{1(1)+2(1)+2(0)}{1^2+1^2+0^2}[1, 1, 0]^T = \frac{3}{2}[1, 1, 0]^T = [\frac{3}{2}, \frac{3}{2}, 0]^T$. Then, $\mathbf{v}_2 = \mathbf{a}_2 - \mathrm{proj}_{\mathbf{v}_1}(\mathbf{a}_2) = [1, 2, 2]^T - [\frac{3}{2}, \frac{3}{2}, 0]^T = [-\frac{1}{2}, \frac{1}{2}, 2]^T$.
13. Let $A = [\mathbf{a}_1, \mathbf{a}_2]$ have the QR factorization $A=QR$. In this context, what is the geometric interpretation of the vector $r_{12}\mathbf{q}_1$?
    Answer: The projection of the vector $\mathbf{a}_2$ onto the subspace spanned by $\mathbf{q}_1$. From the definition of the QR factorization, $r_{12} = \mathbf{q}_1^T \mathbf{a}_2$. Since $\mathbf{q}_1$ is a unit vector ($\!\|\mathbf{q}_1\!\|=1$), this dot product is the scalar projection of $\mathbf{a}_2$ onto $\mathbf{q}_1$. Multiplying this scalar by the unit vector $\mathbf{q}_1$ gives the vector projection of $\mathbf{a}_2$ onto the line (subspace) spanned by $\mathbf{q}_1$. This is a key step in the Gram-Schmidt process, where this projection is subtracted from $\mathbf{a}_2$ to find a vector orthogonal to $\mathbf{q}_1$.
14. Let $A = [\mathbf{a}_1, \mathbf{a}_2]$ be a matrix with QR factorization $A=QR$. A new matrix $A'$ is formed by scaling the second column: $A' = [\mathbf{a}_1, 3\mathbf{a}_2]$. If the QR factorization of $A'$ is $A' = Q'R'$, how do $Q'$ and $R'$ relate to $Q$ and $R$?
    Answer: $Q'=Q$, and the second column of $R'$ is 3 times the second column of $R$. The first step of the Gram-Schmidt process depends only on $\mathbf{a}_1$, so $\mathbf{q}'_1 = \mathbf{q}_1$ and $r'_{11} = r_{11}$. We have $\mathbf{a}_1 = r_{11}\mathbf{q}_1$ and $\mathbf{a}_2 = r_{12}\mathbf{q}_1 + r_{22}\mathbf{q}_2$. For $A'$, we have $A' = [\mathbf{a}_1, 3\mathbf{a}_2] = [r_{11}\mathbf{q}_1, 3(r_{12}\mathbf{q}_1 + r_{22}\mathbf{q}_2)] = [r_{11}\mathbf{q}_1, (3r_{12})\mathbf{q}_1 + (3r_{22})\mathbf{q}_2]$. This can be written as $A' = [\mathbf{q}_1, \mathbf{q}_2] \begin{pmatrix} r_{11} & 3r_{12} \\ 0 & 3r_{22} \end{pmatrix}$. Thus, $Q'=Q$ and the second column of $R'$ is 3 times the second column of $R$.
15. In the QR factorization $A=QR$, the Gram-Schmidt process is used to generate an orthogonal set of vectors $\{\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_n\}$ from the columns of $A$. How are the diagonal entries $r_{kk}$ of the matrix $R$ related to these vectors?
    Answer: $r_{kk} = \|\mathbf{v}_k\|$ The columns of $Q$ are the normalized versions of the orthogonal vectors $\mathbf{v}_k$, so $\mathbf{q}_k = \frac{\mathbf{v}_k}{\|\mathbf{v}_k\|}$. The relationship between the columns of $A$ and the columns of $Q$ is given by $\mathbf{a}_k = r_{1k}\mathbf{q}_1 + \dots + r_{kk}\mathbf{q}_k$. The Gram-Schmidt process defines $\mathbf{v}_k$ as the component of $\mathbf{a}_k$ orthogonal to the subspace spanned by $\{\mathbf{q}_1, \dots, \mathbf{q}_{k-1}\}$. This component is $r_{kk}\mathbf{q}_k$. Therefore, $\mathbf{v}_k = r_{kk}\mathbf{q}_k$, and taking the norm of both sides gives $\|\mathbf{v}_k\| = |r_{kk}| \|\mathbf{q}_k\| = |r_{kk}|$. By convention, $r_{kk}$ is chosen to be positive, so $r_{kk} = \|\mathbf{v}_k\|$.
16. Consider the matrix $A = \begin{pmatrix} 0 & 3 \\ 1 & 5 \\ 0 & 4 \end{pmatrix}$. Let $A=QR$ be its QR factorization. What are the first column of $Q$, denoted $\mathbf{q}_1$, and the entry $r_{11}$ of $R$?
    Answer: $\mathbf{q}_1 = [0, 1, 0]^T$ and $r_{11} = 1$ The first step in the Gram-Schmidt process is to set $\mathbf{v}_1 = \mathbf{a}_1 = [0, 1, 0]^T$. Next, we find the norm of $\mathbf{v}_1$, which is $\|\mathbf{v}_1\| = \sqrt{0^2+1^2+0^2} = 1$. The first diagonal entry of $R$ is this norm, so $r_{11} = 1$. The first column of $Q$ is found by normalizing $\mathbf{v}_1$: $\mathbf{q}_1 = \frac{\mathbf{v}_1}{\|\mathbf{v}_1\|} = \frac{[0, 1, 0]^T}{1} = [0, 1, 0]^T$. Since the first column of $A$ was already a unit vector, the process is simplified for the first step.
17. Consider the matrix $A = \begin{pmatrix} 2 & 4 \\ 0 & 3 \\ 0 & 0 \end{pmatrix}$. In any QR factorization $A = QR$ of this matrix, what can be concluded about the matrix $R$?
    Answer: $R$ must be a $2 \times 2$ matrix with positive diagonal entries Since $A$ has rank 2 (two linearly independent columns), the reduced QR factorization gives $Q$ as $3 \times 2$ and $R$ as $2 \times 2$ upper triangular. The diagonal entries of $R$ are the norms of the orthogonalized vectors, which are positive. Choice A describes the wrong dimension of $R$. Choice B is incorrect because the columns of $A$ are not orthonormal (the first column has norm 2, not 1). Choice C is wrong because $R_{11} = ||\mathbf{a}_1|| = 2$ but $R_{22}$ equals the norm of the component of $\mathbf{a}_2$ orthogonal to $\mathbf{a}_1$, which is not 3.
18. The QR factorization of a matrix $A$ with linearly independent columns is $A=QR$. Under which condition is this factorization guaranteed to be unique?
    Answer: The factorization is unique if the diagonal entries of $R$ are required to be positive. The QR factorization is not strictly unique because of sign choices. If we have a factorization $A = QR$, we can define a diagonal matrix $D$ with entries $\pm 1$. Then $A = (QD)(D^{-1}R) = Q'R'$ is another valid QR factorization. The matrix $Q'$ still has orthonormal columns, and $R'$ is still upper triangular. To ensure uniqueness, a convention is adopted: requiring all diagonal entries of $R$ to be positive. This constraint fixes the signs of the columns in $Q$.
19. A $4 \times 3$ matrix $A$ has the property that its first two columns are orthogonal unit vectors, and its third column is orthogonal to the first column but not to the second. In the QR factorization $A = QR$, which entry of $R$ represents the component of the third column of $A$ in the direction of the second column of $A$?
    Answer: $R_{23}$ When you encounter QR factorization questions, remember that the process systematically orthogonalizes columns using the Gram-Schmidt procedure, and the $R$ matrix stores the "memory" of how each original column relates to the orthogonal basis being built. In QR factorization, each entry $R_{ij}$ represents how the $j$-th original column projects onto the $i$-th orthogonal vector. Since the first two columns of $A$ are already orthogonal unit vectors, they become the first two columns of $Q$ directly. When processing the third column, the algorithm computes its projections onto all previous orthogonal vectors. The component of the third column in the direction of the second column is exactly what $R_{23}$ captures - this is the projection coefficient of column 3 onto the second orthogonal basis vector. This makes **(D) $R_{23}$** correct. **(A) $R_{31}$ would represent how a hypothetical fourth column projects onto the first basis vector, but we only have three columns. (B) $R_{32}$** reverses the indices - this entry doesn't exist in the standard QR decomposition since $R$ is upper triangular. **(C) $R_{13}$** represents how the third column projects onto the first column, not the second. Study tip: In $R_{ij}$, think "column $j$ projected onto basis vector $i$." The upper triangular structure means $i \leq j$, and each column index corresponds to an original column of $A$ while each row index corresponds to a basis vector in $Q$.

Start this quiz