Matrix decompositions

Matrix factorisations play a key role in the solution of problems of the type Ax=bA x = b. Often (e.g. ODE solvers), you have a fixed matrix AA that must be solved with many different bb vectors. A matrix factorisation is effectivly a pre-processing step that allows you to partition AA into multiple factors (e.g. A=LUA = LU in the case of LULU decomposition), so that the actual solve is as quick as possible. Different decompositions have other uses besides solving Ax=bA x = b, for example:

  • the LULU, QRQR and Cholesky decomposition can be used to quickly find the determinant of a large matrix, since det⁡(AB)=det⁡(A)det⁡(B)\det(AB) = \det(A) \det(B) and the determinant of a triangular matrix is simply the product of its diagonal entries.
  • The Cholesky decomposition can be used to sample from a multivariate normal distribution, and is a very efficient technique to solve Ax=bA x = b for the specific case of a positive definite matrix.
  • The QRQR decomposition can be used to solve a minimum least squares problem, to find the eigenvalues and eigenvectors of a matrix, and to calulcate the Singular Value Decomposition (SVD), which is itself another very useful decomposition!