Determinants

Given a \(2 \times 2\) matrix with the upper left entry nonzero, we can simplify the matrix to be upper triangular like so:

\begin{align} \begin{bmatrix} a & b \\ c & d \end{bmatrix} \sim \begin{bmatrix} a & b \\ ac & ad \end{bmatrix} \sim \begin{bmatrix} a & b \\ 0 & ad-bc \end{bmatrix} \notag \end{align}

We want \(ad-bc \neq 0\). Thus, we call \(ad-bc\) the determinant of a \(2 \times 2\) matrix, where the matrix is invertible if and only if \(ad-bc \neq 0\).

We can also try to turn a \(3 \times 3\) matrix into an upper triangular form, if the top left entry is nonzero (this can always be the case, unless the first entry for all the rows are zero, in which case it is clear that the matrix is not invertible):

\begin{align} \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{bmatrix} \sim \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{11}a_{21} & a_{11}a_{22} & a_{11}a_{23} \\ a_{11}a_{31} & a_{11}a_{32} & a_{11}a_{33} \end{bmatrix} \sim \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ 0 & a_{11}a_{22} - a_{12}a_{21} & a_{11}a_{23}-a_{13}a_{21} \\ 0 & a_{11}a_{32} - a_{12}a_{31} & a_{11}a_{33} - a_{13}a_{31} \end{bmatrix} \notag \end{align}

Since we know that \(a_{11}\) is nonzero, we want to know if the diagonal of the \(2 \times 2\) matrix formed by removing the first column and row have entries that are all nonzero. However, this is exactly what the determinant does, so we can just find the determinant of that submatrix.

Therefore, calculating this determinant gives us:

\begin{align} (a_{11}a_{22} - a_{12}a_{21})(a_{11}a_{33} - a_{13}a_{31}) - (a_{11}a_{23}-a_{13}a_{21})(a_{11}a_{32} - a_{12}a_{31}) \neq 0 \notag \\ a_{11}^{2}a_{22}a_{33}-a_{11}a_{12}a_{21}a_{33}-a_{11}a_{13}a_{22}a_{31}+a_{12}a_{13}a_{21}a_{31}-[a_{11}^{2}a_{23}a_{32}-a_{11}a_{13}a_{21}a_{32}-a_{11}a_{12}a_{23}a_{31}+a_{12}a_{13}a_{21}a_{31}] \neq 0 \notag \\ a_{11}[a_{11}a_{22}a_{33}+a_{13}a_{21}a_{32}+a_{12}a_{23}a_{31}-a_{12}a_{21}a_{33}-a_{13}a_{22}a_{31}-a_{11}a_{23}a_{32}]+a_{12}a_{13}a_{21}a_{31}-a_{12}a_{13}a_{21}a_{31} \neq 0\notag \\ a_{11}[a_{11}a_{22}a_{33}+a_{13}a_{21}a_{32}+a_{12}a_{23}a_{31}-a_{12}a_{21}a_{33}-a_{13}a_{22}a_{31}-a_{11}a_{23}a_{32}] \neq 0 \notag \\ a_{11}a_{22}a_{33}+a_{13}a_{21}a_{32}+a_{12}a_{23}a_{31}-a_{12}a_{21}a_{33}-a_{13}a_{22}a_{31}-a_{11}a_{23}a_{32} \neq 0 \notag \end{align}

This is precisely the determinant of a \(3 \times 3\) matrix. We can view this, alternatively, as:

\begin{align} \det(A) = a_{11} \begin{vmatrix} a_{22} & a_{23} \\ a_{32} & a_{33} \end{vmatrix} - a_{12} \begin{vmatrix} a_{21} & a_{23} \\ a_{31} & a_{33} \end{vmatrix} + a_{13} \begin{vmatrix} a_{21} & a_{22} \\ a_{31} & a_{32} \end{vmatrix} \end{align}

We can generalize our method to any \(n \times n\) matrix \(A=[a_{ij}]\). We call the determinant of the submatrix with the i-th row and j-th column removed the minor of an entry \(a_{ij}\), or \(M_{ij}\). Then, define the cofactor \(C_{ij}\) of entry \(a_{ij}\) to be \((-1)^{i+j}M_{ij}\).

We claim that we can apply cofactor expansion to any row and column to find the determinant like so:

\begin{align} \det(A) = \sum a_{ij}C_{ij} \end{align}

where \(i\) and \(j\) iterate along some row or column of \(A\).

We want to consider the general determinant of an upper triangular matrix, like so:

\begin{align} A = \begin{vmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ 0 & a_{22} & \cdots & a_{2n} \\ \vdots & & \ddots \\ 0 & 0 & \cdots & a_{nn} \end{vmatrix} \notag \end{align}

If we do cofactor expansion on the first column, notice that we only need to consider \(a_{11}\) because all the other entries are \(0\). Thus, the cofactor expansion of this matrix is just:

\begin{align} \det A &= a_{11} \begin{vmatrix}a_{22} & \cdots & a_{2n} \\ 0 & \ddots \\ 0 & \cdots & a_{nn} \end{vmatrix} \notag \\ &= a_{11} \cdot a_{22} \begin{vmatrix}a_{32} & \cdots & a_{2n} \\ 0 & \ddots \\ 0 & \cdots & a_{nn}\end{vmatrix} \notag \\ &\vdots \notag \\ &= a_{11} \cdot a_{22} \cdot \cdots \cdot a_{nn} \end{align}

So, the determinant of a triangular matrix is just the product of the diagonal entries.

Since the identity matrix is in triangular form, with ones on the diagonal and zeroes everywhere else, the determinant of any identity matrix is just \(1\).

A property of the product of determinants, if \(A, B\) are square matrices:

\begin{align} \det (AB) = \det A \cdot \det B \end{align}

Consider the \(2 \times 2\) case:

\begin{align} \begin{vmatrix} a & b \\ c & d \end{vmatrix} &= ad - bc \notag \\ \begin{vmatrix} ra & rb \\ c & d \end{vmatrix} &= rad - rbc = r(ad - bc) \notag \end{align}

We see that scaling a row in a matrix by \(r\) scales the determinant by \(r\) as well. This can be generalized to any size square matrix as by cofactor expansion, their determinant can all be represented by a sum of \(2 \times 2\) determinants, whose \(r\) can then be factored out.

Consider the \(2 \times 2\) case:

\begin{align} \begin{vmatrix} a & b \\ c & d \end{vmatrix} &= ad - bc \notag \\ \begin{vmatrix} c & d \\ a & b \end{vmatrix} &= bc - ad \notag \end{align}

We see that interchanging a row makes the determinant the negative of the original. To see why this applies to any size square matrix, consider the following two cases.

In the first case, we interchange two adjacent rows. We can keep the minors in the cofactor expansion the same by simply swapping which row to expand on. However, the only thing that changes is the sign of \((-1)^{i+j}\): since \(i\) changes by one, the determinant changes by a factor of \(-1\). In other words, interchanging two adjacent rows in a square matrix makes the new determinant the negative of the original.

The second case would be interchanging two non-adjacent rows. However, realize that we reach the same result of interchanging two non-adjacent rows by chaining together interchanges of adjacent rows. Let's say their are \(n\) rows between the two non-adjacent rows we want to interchange. It can be shown that it takes \(n+1\) interchanges to get the first row to the place of the second row, and another \(n\) interchanges to get the second row back to the place of the first row. Therefore, in total, there will always be \(2n+1\) adjacent row interchanges, which is always an odd number. Since multiplying the original determinant by a factor of \({-1}^{2n+1}\) will always yield \(-1\), interchanging two non-adjacent rows will also make the new determinant the negative of the original.

Consider the following \(3 \times 3\) matrix where the first row has been added to a multiple of the second row:

\begin{align} \begin{vmatrix} a_{11} + ra_{21} & a_{12} + ra_{22} & a_{13} + ra_{23} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{vmatrix} = \begin{vmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{vmatrix} + \begin{vmatrix} ra_{21} & ra_{22} & ra_{23} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{vmatrix} \notag \end{align}

We can break this up into these two matrices by considering how cofactor expansion on the first row works: the minors are the same, and we can break apart each of the entries and distribute.

Notice that the first matrix is simply the determinant of the original matrix, and the second matrix has the first row as a multiple of the second row. This means the determinant of the second matrix must be \(0\), since if one row is the multiple of another, it is impossible to have a pivot in every row and thus the matrix is not invertible.

In other words, the determinant of a matrix that has one row added to the multiple of another is just the determinant of the original matrix.

Consider \(A \cdot I_i(\mathbf{x})\). We know that due to matrix multiplication by columns, we have the following:

\begin{align} AI_i(\mathbf{x}) &= A_i(\mathbf{b}) \notag \\ \det[AI_i(\mathbf{x})] &= \det[A_i(\mathbf{b})] \notag \\ \det A \cdot \det I_i(\mathbf{x}) &= \det[A_i(\mathbf{b})] \notag \\ \det A \cdot x_i &= \det[A_i(\mathbf{b})] \notag \\ x_i &= \frac{\det[A_i(\mathbf{b})]}{\det A} \notag \end{align}