Normal Equations - Linear Algebra
- How are the normal equations derived from orthogonality?
- The residual is orthogonal to every column of , so .
- For and , what is ?
- . These are and .
- When is invertible?
- Exactly when the columns of are linearly independent.
- How is the null space of related to the null space of ?
- They are equal: .
- If is , what is the size of the normal-equation matrix ?
- . is , so is .
- What structural properties does always have?
- It is square ( for of size ), symmetric, and positive semidefinite.
- For , what is ?
- . The entries are dot products of the columns and .
- is the single column and . Solve the normal equation for .
- . The equation is .
- What does the word normal refer to in the normal equations?
- Perpendicular: the equations say the residual is normal (orthogonal) to .
- What are the normal equations for ?
- . Their solutions are exactly the least-squares solutions.
- Solve the normal equations .
- . Check: and .
- When is invertible, what is the formula for the least-squares solution?
- .