Subsection 2.2.4 Unitary matrices
¶Homework 2.2.4.1.
Let \(Q \in \C^{m \times n} \) be an orthonormal matrix.
ALWAYS/SOMETIMES/NEVER: \(Q^{-1} = Q^H \) and \(Q Q^H = I \text{.}\)
SOMETIMES
Now explain it!
If \(Q \) is unitary, then it is an orthonormal matrix and square. Because it is an orthonormal matrix, \(Q^H Q = I \text{.}\) If \(A, B \in \C^{m \times m} \text{,}\) the matrix \(B \) such that \(B A = I \) is the inverse of \(A \text{.}\) Hence \(Q^{-1} = Q^H \text{.}\) Also, if \(B A = I \) then \(A B = I \) and hence \(Q Q^H = I \text{.}\)
However, an orthonormal matrix is not necessarily square. For example, the matrix \(Q = \left( \begin{array}{c} \frac{\sqrt{2}}{2} \\ \frac{\sqrt{2}}{2}\end{array} \right) \) is an orthonormal matrix: \(Q^T Q = I \text{.}\) However, it doesn't have an inverse because it is not square.
If an orthonormal matrix is square, then it is called a unitary matrix.
Definition 2.2.4.1. Unitary matrix.
Let \(U \in \C^{m \times m} \text{.}\) Then \(U \) is said to be a unitary matrix if and only if \(U^H U = I \) (the identity).
Remark 2.2.4.2.
Unitary matrices are always square. Sometimes the term orthogonal matrix is used instead of unitary matrix, especially if the matrix is real valued.
Unitary matrices have some very nice properties, as captured by the following exercises.
Homework 2.2.4.2.
Let \(Q \in \C^{m \times m} \) be a unitary matrix.
ALWAYS/SOMETIMES/NEVER: \(Q^{-1} = Q^H \) and \(Q Q^H = I \text{.}\)
Homework 2.2.4.3.
TRUE/FALSE: If \(U \) is unitary, so is \(U^H \text{.}\)
Clearly, \(U^H \) is square. Also, \(( U^H )^H U^H = ( U U^H )^H = I \) by the last homework.
Homework 2.2.4.4.
Let \(U_0, U_1 \in \C^{m \times m} \) both be unitary.
ALWAYS/SOMETIMES/NEVER: \(U_0 U_1 \text{,}\) is unitary.
Homework 2.2.4.5.
Let \(U_0, U_1, \ldots, U_{k-1} \in \C^{m \times m} \) all be unitary.
ALWAYS/SOMETIMES/NEVER: Their product, \(U_0 U_1 \cdots U_{k-1} \text{,}\) is unitary.
ALWAYS
Now prove it!
Strictly speaking, we should do a proof by induction. But instead we will make the more informal argument that
\begin{equation*}\begin{array}{rcl}( U_0 U_1 \cdots U_{k-1})^H U_0 U_1 \cdots U_{k-1}\amp=\ampU_{k-1}^H \cdots U_1^H U_0^HU_0 U_1 \cdots U_{k-1} \\\amp=\amp\begin{array}[t]{c}\underbrace{U_{k-1}^H \begin{array}[t]{c}\underbrace{\cdots \begin{array}[t]{c}\underbrace{U_1^H \begin{array}[t]{c}\underbrace{U_0^HU_0}\\I\end{array}U_1 }\\I\end{array}\cdots }\\I\end{array}U_{k-1} }\\I\end{array}= I.\end{array}\end{equation*}
(When you see a proof that involed \(\cdots \text{,}\) it would be more rigorous to use a proof by induction.)
Remark 2.2.4.3.
Many algorithms that we encounter in the future will involve the application of a sequence of unitary matrices, which is why the result in this last exercise is of great importance.
Perhaps the most important property of a unitary matrix is that it preserves length.
Homework 2.2.4.6.
Let \(U \in \Cmxm \) be a unitary matrix and \(x \in\Cm \text{.}\) Prove that \(\| U x \|_2 = \| x \|_2 \text{.}\)
\begin{equation*}\begin{array}{l}\| U x \|_2^2 \\~~~=~~~~ \lt \mbox{ alternative definition } \gt \\( U x )^H U x \\~~~=~~~~\lt ( A z )^H = z^H A^H \gt \\x^H U^H U x \\~~~=~~~~ \lt U \mbox{ is unitary } \gt \\x^H x \\~~~=~~~~ \lt \mbox{ alternative definition } \gt \\\| x \|_2^2 .\end{array}\end{equation*}
The converse is true as well:
Theorem 2.2.4.4.
If \(A \in \Cmxm \) preserves length (\(\| A x \|_2 = \| x \|_2 \) for all \(x \in \Cm \)), then \(A \) is unitary.
Proof.
We first prove that \(( A x )^H ( A y ) = x^H y \) for all \(x, y \) by considering \(\| x - y \|_2^2 = \| A( x - y ) \|_2^2 \text{.}\) We then use that to evaluate \(e_i^H A^H A e_j \text{.}\)
Let \(x, y \in \Cm \text{.}\) Then
\begin{equation*}\begin{array}{l}\| x - y \|_2^2 = \| A ( x - y ) \|_2^2 \\~~~ \Leftrightarrow ~~~~ \lt \mbox{ alternative definition } \gt \\( x - y )^H ( x - y ) = ( A ( x - y ) )^H A ( x - y ) \\~~~ = ~~~~ \lt ( B z )^H = z^H B^H \gt \\( x - y )^H ( x - y ) = ( x - y )^H A^H A ( x - y ) \\~~~ \Leftrightarrow ~~~~ \lt \mbox{ multiply out } \gt \\ x^H x - x^H y - y^H x + y^H y = x^H A^H A x - x^H A^H Ay - y^H A^H A x + y^H A^H A y \\~~~ \Leftrightarrow ~~~~ \lt \mbox{ alternative definition };\overline {x^H y } = y^H x \gt \\\| x \|_2^2 - ( x^H y + \overline {x^H y} ) + \| y \|_2^2 =\| A x \|_2^2 - ( x^H A^H A y + \overline {x^H A^H A y} ) + \| A y \|_2^2 \\~~~\Leftrightarrow~~~~ \lt \| A x \|_2 = \| x \|_2 \mbox{ and } \| A y\|_2 = \| y \|_2; \alpha + \overline \alpha = 2 {\rmRe}( \alpha ) \gt \\{\rm Re}\left( x^H y \right) ={\rm Re}\left( (A x )^H A y \right) \end{array}\end{equation*}
One can similarly show that \({\rm Im}\left( x^H y \right) ={\rm Im}\left( (A x )^H A y \right) \) by considering \(A( i x - y )\text{.}\)
Conclude that \(( A x )^H (A y) = x^H y\text{.}\)
We now use this to show that \(A^H A = I \) by using the fact that the standard basis vectors have the property that
\begin{equation*}e_i^H e_j = \left\{ \begin{array}{l l}1 \amp \mbox{if } i = j \\0 \amp \mbox{ otherwise}\end{array}\right.\end{equation*}
and that the \(i,j\) entry in \(A^H A \) equals \(e_i^H A^H A e_j \text{.}\)
Note: I think the above can be made much more elegant by choosing \(\alpha \) such that \(\alpha x^H y \) is real and then looking at \(\| x + \alpha y \|_2 = \| A ( x +\alpha y ) \|_2 \) instead, much like we did in the proof of the Cauchy-Schwartz inequality. Try and see if you can work out the details.
Homework 2.2.4.7.
Prove that if \(U \) is unitary then \(\| U \|_2 = 1 \text{.}\)
\begin{equation*}\begin{array}{l}\| U \|_2 \\~~~ = ~~~~ \lt \mbox{ definition } \gt \\\max_{\| x \|_2 = 1 } \| U x \|_2 \\~~~ = ~~~~ \lt \mbox{ unitary matrices preserve length } \gt \\\max_{\| x \|_2 = 1 } \| x \|_2 \\~~~ = ~~~~ \lt \mbox{ algebra } \gt \\1 \end{array}\end{equation*}
(The above can be really easily proven with the SVD. Let's point that out later.)
Homework 2.2.4.8.
Prove that if \(U \) is unitary then \(\kappa_2( U )= 1\text{.}\)
\begin{equation*}\begin{array}{l}\kappa_2{ U } \\~~~ = ~~~~ \lt \mbox{ definition } \gt \\\| U \|_2 \| U^{-1} \|_2 \\~~~ = ~~~~ \lt \mbox{ both } U \mbox{ and } U^{-1} \mbox{ are unitary }; \mbox{ last homework } \gt \\1 \times 1 \\~~~ = ~~~~ \lt \mbox{ arithmetic } \gt \\1 \end{array}\end{equation*}
The preservation of length extends to the preservation of norms that have a relation to the 2-norm:
dHomework 2.2.4.9.
Let \(U \in \Cmxm \) and \(V \in \Cnxn \) be unitary and \(A \in \Cmxn \text{.}\) Show that
\(\| U^H A \|_2 = \| A \|_2 \text{.}\)
\(\| A V \|_2 = \| A \|_2 \text{.}\)
\(\| U^H A V \|_2 = \| A \|_2 \text{.}\)
Exploit the definition of the 2-norm:
\begin{equation*}\| A \|_2 = \max_{\| x \|_2 = 1} \| A x \|_2. \end{equation*}
\begin{equation*}\begin{array}{l}\| U^H A \|_2 \\~~~=~~~~ \lt \mbox{ definition of 2-norm } \gt \\\max_{\| x \|_2 = 1} \| U^H A x \|_2 \\~~~=~~~~ \lt U \mbox{ is unitary and unitary matrices preserve length } \gt \\\max_{\| x \|_2 = 1} \| A x \|_2 \\~~~=~~~~ \lt \mbox{ definition of 2-norm } \gt \\\| A \|_2.\end{array}\end{equation*}
\begin{equation*}\begin{array}{l}\| A V \|_2 \\~~~=~~~~ \lt \mbox{ definition of 2-norm } \gt \\\max_{\| x \|_2 = 1} \| A V x \|_2 \\~~~=~~~~ \lt V^H \mbox{ is unitary and unitary matrices preserve length } \gt \\\max_{\| V x \|_2 = 1} \| A ( V x ) \|_2 \\~~~=~~~~ \lt \mbox{ substitute } y = V x \gt \\\max_{\| y \|_2 = 1} \| A y \|_2 \\~~~=~~~~ \lt \mbox{ definition of 2-norm } \gt \\\| A \|_2.\end{array}\end{equation*}
The last part follows immediately from the previous two:
\begin{equation*}\| U^H A V \|_2 =\| U ^H(A V) \|_2 =\| A V \|_2 =\| A \|_2.\end{equation*}
Homework 2.2.4.10.
Let \(U \in \Cmxm \) and \(V \in \Cnxn \) be unitary and \(A \in \Cmxn \text{.}\) Show that
\(\| U^H A \|_F = \| A \|_F \text{.}\)
\(\| A V \|_F = \| A \|_F \text{.}\)
\(\| U^H A V \|_F = \| A \|_F \text{.}\)
How does \(\| A \|_F \) relate to the 2-norms of its columns?
Partition
\begin{equation*}A =\left( \begin{array}{c | c | c}a_0 \amp \cdots \amp a_{n-1}\end{array}\right).\end{equation*}
Then we saw in Subsection1.3.3 that \(\| A \|_F^2 =\sum_{j=0}^{n-1} \| a \|_2^2 \text{.}\)
Now,
\begin{equation*}\begin{array}{l}\| U^H A \|_F^2 \\~~~=~~~~ \lt \mbox{ partition } A\mbox{ by columns} \gt \\\| U^H \left( \begin{array}{c | c | c}a_0 \amp \cdots \amp a_{n-1}\end{array}\right) \|_F^2 \\~~~=~~~~ \lt\mbox{ property of matrix-vector multiplication } \gt \\\| \left( \begin{array}{c | c | c}U^H a_0 \amp \cdots \amp U^H a_{n-1}\end{array}\right) \|_F^2 \\~~~=~~~~ \lt \mbox{ exercise in Chapter 1 } \gt \\\sum_{j=0}^{n-1} \| U^H a_j \|_2^2 \\~~~=~~~~ \lt\mbox{ unitary matrices preserve length } \gt \\\sum_{j=0}^{n-1} \| a_j \|_2^2 \\~~~=~~~~ \lt \mbox{ exercise in Chapter 1 } \gt \\\| A \|_F^2.\end{array}\end{equation*}
To prove that \(\| A V \|_F = \| A \|_F \) recall that \(\| A^H \|_F = \| A \|_F \text{.}\)
The last part follows immediately from the first two parts.
In the last two exercises we consider \(U^H A V \) rather than \(U A V \) because it sets us up better for future discussion.
