The whole Linear Algebra formula sheet, to print

Applies to any subset W of a vector space V. If the zero vector is missing, W fails immediately.

W1 and W2 are finite-dimensional subspaces of a vector space V. The intersection and the sum W1+W2 (all u+v with u in W1, v in W2) are always subspaces. The union is a subspace exactly when one of the two contains the other.

The span is always a subspace, and it is the smallest subspace containing the given vectors.

A is m x n real, b is in R^m. Use it to decide whether a vector lies in the span of given vectors.

The vectors are independent if only the trivial combination gives the zero vector; otherwise they are dependent. For k >= 2 this is equivalent to: no vector in the set is a combination of the others. Also equivalent: the list v1, ..., vk is dependent exactly when v1 = 0 or some vj is a combination of v1, ..., v(j-1).

V is finite-dimensional. Any independent set has at most n vectors, any spanning set has at least n, and an independent set can be extended to a basis. If W is a subspace of V, then dim W <= n, with equality only when W = V.

A is real m x n of rank r. Row rank equals column rank, so rank(A) = rank(A^T), and r <= min(m, n). The middle result, rank plus nullity equals the number of columns n, is the rank-nullity theorem.

n is the number of columns of A. Nullity is the dimension of the null space.

Holds for every square real matrix, and the split is unique.

Square real $Q$. Columns (and rows) form an orthonormal set.

Holds for every $\mathbf{x},\mathbf{y}$ when $Q$ is orthogonal.

A real eigenvalue can only be $+1$ or $-1$. Complex eigenvalues come in conjugate pairs of modulus 1.

Every real $2\times 2$ orthogonal matrix is one of these two forms. $R(\theta )$ is a rotation by $\theta$: $\det =+1$, eigenvalues $\cos \theta \pm i\sin \theta$. $F(\theta )$ is a reflection: symmetric, $\det =-1$, eigenvalues $1$ and $-1$.

Square matrix with $A^2=A$. The converse is false: $\mathrm{diag}(3,-1)$ has rank $2=$ trace $2$ and is not idempotent. For a general matrix, rank and trace are unrelated.

If $A$ ($n\times n$) is idempotent, then so is $I-A$, and $\mathrm{rank}(A)+\mathrm{rank}(I-A)=n$.

Needs the $k$ columns of $A$ linearly independent, so that the $k\times k$ matrix $A^{T}A$ is invertible. $P\mathbf{b}$ is the point of the column space nearest to $\mathbf{b}$.

First: projection onto the line through $\mathbf{a}\ne 0$. Second: $Q$ is $n\times k$ with $Q^{T}Q=I_k$, and need not be square.

For $P=A(A^{T}A{)}^{-1}{A}^T$ with $A$ of size $n\times k$ and rank $k$. $P$ has eigenvalue $1$ on the column space (multiplicity $k$) and $0$ on its orthogonal complement (multiplicity $n-k$). $P$ depends only on the column space, not on which basis forms the columns of $A$.

$\mathbf{a}\ne 0$ in ${\mathbb{R}}^n$, and $H$ is $n\times n$. $H=I-2P$, with $P$ the projection onto the line through $\mathbf{a}$. $H$ reflects across the hyperplane perpendicular to $\mathbf{a}$. Its eigenvalues are $-1$ once (along $\mathbf{a}$) and $1$ with multiplicity $n-1$. More generally, $I-2P$ is symmetric and orthogonal for every orthogonal projection $P$.

The column partition of the left factor must match the row partition of the right factor, so that every product such as $AE$ and $BG$ is defined.

Transpose each block and also swap the two off-diagonal blocks.

$A$ and $D$ square. The same holds for block lower triangular and block diagonal matrices. The eigenvalues of the whole matrix are those of $A$ together with those of $D$, with multiplicities.

$A$ and $D$ square. The matrix is invertible exactly when $A$ and $D$ are both invertible.

Requires $A$ square and invertible, and $D$ square. $D-C{A}^{-1}B$ is the Schur complement of $A$. If instead $D$ is invertible, $\det \left[\begin{array}{cc}A& B\\ C& D\end{array}\right]=\det D\cdot \det (A-B{D}^{-1}C)$.

$A$ is real symmetric. Example: $x_1^2+6x_1{x}_2+4x_2^2$ has $A=\left[\begin{array}{cc}1& 3\\ 3& 4\end{array}\right]$.

A real symmetric $A$ always has real eigenvalues ${\lambda }_i$ and an orthogonal $n\times n$ matrix $Q$ whose columns are eigenvectors. $\Lambda =\mathrm{diag}({\lambda }_1,\dots ,{\lambda }_n)$. The signs of the ${\lambda }_i$ decide the definiteness.

$A$ real symmetric. ${\Delta }_k$ is the determinant of the top-left $k\times k$ block. For $n=2$: $\left[\begin{array}{cc}a& b\\ b& c\end{array}\right]$ is positive definite exactly when $a>0$ and $ac-b^2>0$.

$A$ real symmetric. Equality holds when $\mathbf{x}$ is an eigenvector for ${\lambda }_{\max }$ (or for ${\lambda }_{\min }$). It follows from $q=\sum {\lambda }_i{y}_i^2$ with $\Vert \mathbf{y}\Vert =\Vert \mathbf{x}\Vert$.

Applies to any $m\times n$ matrix $A$ after row reduction; the rank is the same in every echelon form.

Equivalent statement: $\mathbf{b}$ lies in the column space of $A$. Always $\mathrm{rank}(A)\le \mathrm{rank}([A\mid \mathbf{b}])\le \mathrm{rank}(A)+1$.

When the system is consistent, ${\mathbf{x}}_p$ is any one particular solution; the general solution adds an arbitrary null-space vector.

$n$ is the number of columns of $A$ (not rows). Nullity equals the number of free variables in an echelon form of $A$.

For $A$ of size $m\times n$ and any product $AB$ that is defined. For real matrices also $\mathrm{rank}(A^{T}A)=\mathrm{rank}(A{A}^T)=\mathrm{rank}(A)$.

First (Sylvester's inequality): $A$ is $m\times n$ and $B$ is $n\times p$. Second: $A$ and $B$ have the same size. Third: $P$ ($m\times m$) and $Q$ ($n\times n$) are invertible. Also, for non-zero $\mathbf{u}\in {\mathbb{R}}^m$ and $\mathbf{v}\in {\mathbb{R}}^n$, the $m\times n$ matrix $\mathbf{u}{\mathbf{v}}^T$ has rank 1.

For a square $n\times n$ matrix, with $I$ the $n\times n$ identity. If the left block cannot be reduced to $I$ (a zero row appears in it), $A$ is singular and has no inverse.

For a square $n\times n$ system with $\det A\ne 0$. $A_i$ is $A$ with its $i$-th column replaced by $\mathbf{b}$.

Always valid for a 2 by 2 matrix.

A is n by n and i is any fixed row. M_ij is the determinant of the (n-1) by (n-1) matrix left after deleting row i and column j. The same works along any column. Expand along the row or column with the most zeros.

A and B are n by n. The inverse rule needs det(A) not equal to 0. Applying the product rule repeatedly gives det(A^k) = (det A)^k for every integer k >= 1.

A is n by n. The factor c comes out of each of the n rows, so the power is n, the size of the matrix.

A is n by n with n >= 2, and adj(A) is the transpose of the cofactor matrix. If det(A) is not 0, then A^{-1} = adj(A)/det(A). Also adj(adj A) = (det A)^{n-2} A for n >= 3, and adj(adj A) = A for n = 2.

In general, the n by n matrix whose i-th row is 1, x_i, x_i^2, ..., x_i^{n-1} has determinant equal to the product of (x_j - x_i) over all i < j. It is non-zero exactly when the x_i are distinct.

A is n by n. The n roots of this equation, counted with multiplicity over the complex numbers, are the eigenvalues. For each root, solve (A - lambda I)x = 0 to find the eigenvectors.

Valid for any 2 by 2 matrix; it is det(A - lambda I) = 0 written out.

c_2 is the sum of the three principal 2 by 2 minors: for each i, delete row i and column i, take the determinant, and add the three results. This is det(A - lambda I) = 0 multiplied through by -1.

A is n by n, and the eigenvalues are counted with algebraic multiplicity. Hence A is singular if and only if 0 is an eigenvalue. Applying the trace rule to A^k gives tr(A^k) = lambda_1^k + ... + lambda_n^k for every integer k >= 1.

A, P and D are n by n and P is invertible. The columns of P are n linearly independent eigenvectors, in the same order as the eigenvalues on the diagonal of D. This factorisation exists if and only if A has n linearly independent eigenvectors.

A is an n by n real symmetric matrix, Q is an n by n orthogonal matrix whose columns are orthonormal eigenvectors, and Lambda is the diagonal matrix of the matching eigenvalues. A real symmetric matrix has real eigenvalues, its eigenvectors for distinct eigenvalues are orthogonal, and it is always diagonalisable by an orthogonal matrix.

A is an n by n real symmetric matrix with smallest eigenvalue lambda_min and largest eigenvalue lambda_max. Each bound is attained at an eigenvector for that eigenvalue, so the maximum of x^T A x over unit vectors (||x|| = 1) is lambda_max and the minimum is lambda_min.

A is n by n, and the constant term c of p is replaced by cI. Every square matrix satisfies its own characteristic equation. For a 2 by 2 matrix: A^2 - tr(A) A + det(A) I = 0.

Line through the origin along $\mathbf{a}\ne 0$; $\mathbf{b}$ and $\mathbf{a}$ are in ${\mathbb{R}}^n$, $P$ is $n\times n$ of rank $1$, and $\mathbf{p}=P\mathbf{b}$.

$A$ is $m\times n$ with linearly independent columns (so $A^{T}A$ is invertible); $P$ is $m\times m$ and projects onto the column space of $A$.

The columns ${\mathbf{q}}_1,\dots ,{\mathbf{q}}_k$ of the $m\times k$ matrix $Q$ are orthonormal ($Q^{T}Q=I_k$); $P$ is $m\times m$ and no inverse is needed.

The normal equations always have a solution; it is unique (and given by the inverse form) only when the columns of $A$ are linearly independent.

Pythagoras applies because the projection and the residual are perpendicular.

For an $m\times k$ matrix $A$ with linearly independent columns this gives $A=QR$: $Q$ is $m\times k$ with orthonormal columns and $R=Q^{T}A$ is $k\times k$ upper triangular with entries $r_{ij}={\mathbf{q}}_i^T{\mathbf{a}}_j$ ($i\le j$) and positive diagonal. The span of the first $j$ vectors is preserved at every step.

Shown for $3\times 3$. Elimination without row exchanges succeeds exactly when the first $n-1$ pivots are non-zero, equivalently when the leading principal minors ${\Delta }_1,\dots ,{\Delta }_{n-1}$ are non-zero; the factorisation with unit lower $L$ is then unique. ($u_{nn}$ may be $0$, which happens exactly when $A$ is singular.)

${\Delta }_k$ is the determinant of the top-left $k\times k$ block of $A$; valid when no row exchanges are needed (${\Delta }_1,\dots ,{\Delta }_{n-1}\ne 0$).

When a zero pivot appears, rows are exchanged; $P$ is an $n\times n$ permutation matrix. A factorisation $PA=LU$ exists for every square $A$, and $\det A=(-1{)}^s{\prod }_i{u}_{ii}$, where $s$ is the number of row exchanges.

Valid when no row exchanges were used; $\det L=1$ because $L$ is unit triangular.

$A$ is symmetric $n\times n$ with ${\Delta }_1,\dots ,{\Delta }_n\ne 0$; then $U=D{L}^T$. $A$ is positive definite exactly when every pivot $u_{kk}>0$ (equivalently every ${\Delta }_k>0$), and then the Cholesky factor $G$ is lower triangular with positive diagonal.

$U$ is $m\times m$ orthogonal (left singular vectors ${\mathbf{u}}_i$), $V$ is $n\times n$ orthogonal (right singular vectors ${\mathbf{v}}_i$), $\Sigma$ is $m\times n$ with the $\min (m,n)$ singular values on its diagonal and zeros elsewhere; $r=\mathrm{rank}A$. Equivalently $A={\sum }_{i=1}^r{\sigma }_i{\mathbf{u}}_i{\mathbf{v}}_i^T$.

$A^{T}A$ ($n\times n$) and $A{A}^T$ ($m\times m$) are symmetric positive semidefinite and have the same non-zero eigenvalues; all their other eigenvalues are $0$. The columns of $V$ are orthonormal eigenvectors of $A^{T}A$ and the columns of $U$ are orthonormal eigenvectors of $A{A}^T$.

Fixes ${\mathbf{u}}_i$ so that its sign agrees with ${\mathbf{v}}_i$; choosing ${\mathbf{u}}_i$ independently from $A{A}^T$ can give a wrong sign.

$\mathbf{u}\in {\mathbb{R}}^m$ and $\mathbf{v}\in {\mathbb{R}}^n$ are non-zero, so $A$ is $m\times n$ of rank $1$; this follows from $A^{T}A=\Vert \mathbf{u}{\Vert }^2\mathbf{v}{\mathbf{v}}^T$.

$\Vert A{\Vert }_2={\max }_{\Vert \mathbf{x}\Vert =1}\Vert A\mathbf{x}\Vert$ is the spectral norm; $\Vert A{\Vert }_F$ is the Frobenius norm (square root of the sum of squared entries).

$A$ is $n\times n$; $A$ is invertible exactly when ${\sigma }_n>0$, and ${\kappa }_2$ is the 2-norm condition number.

Keeps the $k$ largest singular values, $k\le r$; $A_k$ has rank $k$.

$B$ ranges over $m\times n$ matrices; the minimum is attained by $B=A_k$ in both norms; take ${\sigma }_{k+1}=0$ if $k\ge r$.