Inner Product
Table of Contents
1. Inner Product
Certain vector spaces have what is known as an inner product. Such spaces are also called inner product spaces. We define the inner product \(\langle u, v \rangle\) as a type of operation that operates on two vectors \(u\) and \(v\) in some vector space \(V\), such that these operations satisfies a certain set of properties. One type of inner product is the dot product.
Specifically, inner products must satisfy:
- \(\langle \mathbf{u}, \mathbf{v} \rangle = \langle \mathbf{v}, \mathbf{u} \rangle \; \forall \; \mathbf{u}, \mathbf{v} \in V\)
- \(\langle \mathbf{u}+\mathbf{v}, \mathbf{w} \rangle = \langle \mathbf{u}, \mathbf{w} \rangle + \langle \mathbf{v}, \mathbf{w} \rangle\)
- \(\langle c\mathbf{u}, \mathbf{v} \rangle = c\langle \mathbf{u}, \mathbf{v}\rangle\)
- \(\forall \; \mathbf{u} \in V, \; \langle \mathbf{u}, \mathbf{u} \rangle \geq 0\) with \(\langle \mathbf{u}, \mathbf{u} \rangle = 0\) only when \(\mathbf{u} = 0\)
1.1. Extra Properties
Knowledge that we have of dot products can be applied to inner prodcuts as well:
- \(\left | \mathbf{v} \right| ^2 = \langle \mathbf{u}, \mathbf{v} \rangle\)
- \(\mathbf{v}\) is a unit vector if \(\left| \mathbf{v} \right| = 1\)
- Distance between \(\mathbf{u}\) and \(\mathbf{v}\) is \(\left| \mathbf{u}-\mathbf{v}\right|\)
- \(\mathbf{u}\) and \(\mathbf{v}\) are orthogonal if \(\langle \mathbf{u}, \mathbf{v}\rangle = 0\)
2. Cauchy-Schwarz Inequality
The Cauchy-Schwarz inequality states that the inner product of two vectors is less than or equal to the product of their magnitudes:
\begin{align} \boxed{\left | \langle \mathbf{u}, \mathbf{v}\rangle \right | \leq \left|\mathbf{u}\right|\left|\mathbf{v}\right|} \end{align}Note that this is consistent with the dot product, as \(\mathbf{u} \cdot \mathbf{v} = \left|\mathbf{u}\right|\left|\mathbf{v}\right|\cos\theta =\Rightarrow \left|\mathbf{u}\cdot\mathbf{v}\right|\leq\left|\mathbf{u}\right|\cdot\left|\mathbf{v}\right|\).
2.1. Proof
Consider vectors \(\mathbf{u}\) and \(\mathbf{v}\), with \(\mathbf{\hat{v}}\) the projection of \(\mathbf{v}\) on \(\mathbf{u}\). First, we know that \(\left|\mathbf{v}\right| \geq \left|\mathbf{\hat{v}}\right|\) always, because we can decompose \(\mathbf{v}\) into:
\begin{align} \mathbf{v} &= \mathbf{\hat{v}} + \mathbf{z} \notag \\ \langle \mathbf{v}, \mathbf{v} \rangle &= \langle \mathbf{\hat{v}} + \mathbf{z}, \mathbf{\hat{v}} + \mathbf{z} \rangle \notag \\ \left|\mathbf{v}\right|^2 &= \left|\mathbf{\hat{v}}\right|^2 + \left|\mathbf{z}\right|^2 \notag \end{align}Now, recall that \(\mathbf{\hat{v}} = \frac{\langle \mathbf{v},\mathbf{u}\rangle}{\langle\mathbf{u},\mathbf{u}\rangle}\mathbf{u}\). Taking the length of both sides:
\begin{align} \left|\mathbf{\hat{v}}\right| &= \frac{\left|\langle\mathbf{v},\mathbf{u}\rangle\right|}{\left|\mathbf{u}\right|^2}\left|\mathbf{u}\right| \notag \\ \Rightarrow \left|\langle\mathbf{v},\mathbf{u}\rangle\right| &= \left|\mathbf{\hat{v}},\mathbf{u}\right|\leq\left|\mathbf{v}\right|\cdot\left|\mathbf{u}\right| \notag \end{align}