Matrix decompositions
Matrix factorisations play a key role in the solution of problems of the type $A x = b$.
Often (e.g. ODE solvers), you have a fixed matrix $A$ that must be solved with many
different $b$ vectors. A matrix factorisation is effectivly a pre-processing step that
allows you to partition $A$ into multiple factors (e.g. $A = LU$ in the case of $LU$
decomposition), so that the actual solve is as quick as possible. Different
decompositions have other uses besides solving $A x = b$, for example:
- the $LU$, $QR$ and Cholesky decomposition can be used to quickly find the determinant
of a large matrix, since $\det(AB) = \det(A) \det(B)$ and the determinant of a
triangular matrix is simply the product of its diagonal entries.
- The Cholesky decomposition can be used to sample from a multivariate normal
distribution,
and is a very efficient technique to solve $A x = b$ for the specific case of a
positive definite matrix.
- The $QR$ decomposition can be used to solve a minimum least squares problem, to find
the eigenvalues and eigenvectors of a matrix, and to calulcate the Singular Value
Decomposition (SVD),
which is itself another very useful decomposition!