Orthogonal projections and the least squares method for approximating solutions from a college linear algebra course.
30 cards · basic cards · AI-written, checked twice. Edit anything.
- Define the orthogonal projection of vector b onto vector a.
Show answer
proj_a(b) = (a . b / a . a) * a- What is an orthogonal projection matrix P?
Show answer
A matrix such that P*v gives the projection of v onto a subspace, with P = P^2 (idempotent) and P^T = P (symmetric).- Define the orthogonal complement of a subspace W.
Show answer
The set of all vectors in R^n orthogonal to every vector in W, denoted W^perp.- State the Orthogonal Decomposition Theorem.
Show answer
Any vector y can be uniquely written as y = y_hat + z, where y_hat is in W and z is in W^perp.- What is the least squares problem?
Show answer
Find a vector x that minimizes ||b - Ax||^2, where Ax is typically in the column space of A.- When does the least squares method apply?
Show answer
When the system Ax = b is overdetermined (more equations than unknowns) or inconsistent, so no exact solution exists.- State the Normal Equations for least squares.
Show answer
A^T * A * x = A^T * b- What defines an orthonormal set of vectors?
Show answer
A set of vectors where each vector has unit length and every pair is orthogonal to each other.- Why is an orthonormal basis computationally advantageous?
Show answer
The projection onto an orthonormal basis is easy to compute: proj_v(b) = (b . v) * v for each basis vector v, without dividing by squared norms.- What is the Gram-Schmidt process?
Show answer
An algorithm that transforms a set of linearly independent vectors into an orthogonal (or orthonormal) set spanning the same subspace.- What is QR factorization?
Show answer
A factorization A = Q * R where Q has orthonormal columns and R is upper triangular, often computed via Gram-Schmidt.- How does QR factorization solve least squares?
Show answer
From A = Q * R and normal equations A^T * A * x = A^T * b, we get R * x = Q^T * b, which is easy to solve since R is triangular.- What is the residual in a least squares problem?
Show answer
The vector r = b - Ax, which is the error or leftover when Ax is the approximation to b.- State the Best Approximation Theorem for least squares.
Show answer
The least squares solution x* gives A*x* as the closest point in the column space of A to b, minimizing ||b - A*x*||.- How do you find the projection of b onto the column space of A?
Show answer
The projection is A * x*, where x* is the least squares solution to Ax = b.
FlashKeepers