Lesson 7 of 10Article18 min
Orthogonality, Projections & Least Squares
When Ax = b has no exact solution, minimise ‖Ax − b‖. The residual b − Ax̂ is orthogonal to Col(A), giving the normal equations.
Best approximate solutions
When Ax = b has no exact solution, minimise ‖Ax − b‖. The residual b − Ax̂ is orthogonal to Col(A), giving the normal equations.
- Gram–Schmidt builds orthonormal bases from independent sets.
- QR factorisation: A = QR with Q orthonormal, R upper triangular.
- Least squares is the linear algebra behind simple regression.