This subsection is optional. It requires material from the prior, also optional, subsection. The work done here will only be needed in the final two sections of Chapter Five.
The prior subsection suggests
that projecting onto the line spanned by
decomposes a vector
into two parts
|
|
that are orthogonal and so are "not interacting".
We will now develop that suggestion.
- Theorem 2.2
If the vectors in a set
are mutually orthogonal and nonzero then that set is linearly independent.
- Corollary 2.3
If the vectors in a size
subset of a
dimensional space are mutually orthogonal and nonzero then that set is a basis for the space.
- Proof
Any linearly independent size
subset of a
dimensional space is a basis.
Of course, the converse of Corollary 2.3
does not hold— not every basis of every subspace
of
is made of mutually orthogonal vectors.
However, we can get the partial converse
that for every subspace of
there is at least one basis
consisting of mutually orthogonal vectors.
- Example 2.4
The members
and
of this basis for
are not orthogonal.
|
|
However, we can derive from
a new basis for the same space that does have mutually orthogonal members.
For the first member of the new basis we simply use
.
![{\displaystyle {\vec {\kappa }}_{1}={\begin{pmatrix}4\\2\end{pmatrix}}}](http://178.128.105.246/host-https-wikimedia.org/api/rest_v1/media/math/render/svg/5f6f1fbca6f82ff56833de51a85ad002abade7b6)
For the second member of the new basis,
we take away from
its part in the direction of
,
|
|
which leaves the part,
pictured above, of
that is orthogonal to
(it is orthogonal by the definition of the projection onto the span of
). Note that, by the corollary,
is a basis for
.
- Definition 2.5
An orthogonal basis for a vector space is a basis of mutually orthogonal vectors.
- Example 2.6
To turn this basis for
![{\displaystyle \left\langle {\begin{pmatrix}1\\1\\1\end{pmatrix}},{\begin{pmatrix}0\\2\\0\end{pmatrix}},{\begin{pmatrix}1\\0\\3\end{pmatrix}}\right\rangle }](http://178.128.105.246/host-https-wikimedia.org/api/rest_v1/media/math/render/svg/0a00bf8b819e3f0e104ffe48e55f152e2c2da0af)
into an orthogonal basis, we take the first vector as it is given.
![{\displaystyle {\vec {\kappa }}_{1}={\begin{pmatrix}1\\1\\1\end{pmatrix}}}](http://178.128.105.246/host-https-wikimedia.org/api/rest_v1/media/math/render/svg/b13717d5d447a92098660f557c1a16b4dba7ddb4)
We get
by starting with the given second vector
and
subtracting away the part of it in the direction of
.
![{\displaystyle {\vec {\kappa }}_{2}={\begin{pmatrix}0\\2\\0\end{pmatrix}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\begin{pmatrix}0\\2\\0\end{pmatrix}})={\begin{pmatrix}0\\2\\0\end{pmatrix}}-{\begin{pmatrix}2/3\\2/3\\2/3\end{pmatrix}}={\begin{pmatrix}-2/3\\4/3\\-2/3\end{pmatrix}}}](http://178.128.105.246/host-https-wikimedia.org/api/rest_v1/media/math/render/svg/efaef6f72068cd400dba6470e842cd8c0e5f4caf)
Finally, we get
by taking the third given vector and subtracting the part of it
in the direction of
, and also the part
of it in the direction of
.
![{\displaystyle {\vec {\kappa }}_{3}={\begin{pmatrix}1\\0\\3\end{pmatrix}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\begin{pmatrix}1\\0\\3\end{pmatrix}})-{\mbox{proj}}_{[{\vec {\kappa }}_{2}]}({\begin{pmatrix}1\\0\\3\end{pmatrix}})={\begin{pmatrix}-1\\0\\1\end{pmatrix}}}](http://178.128.105.246/host-https-wikimedia.org/api/rest_v1/media/math/render/svg/5305a998b025fa73d4e7c301c77d80a29bce293c)
Again the corollary gives that
![{\displaystyle \left\langle {\begin{pmatrix}1\\1\\1\end{pmatrix}},{\begin{pmatrix}-2/3\\4/3\\-2/3\end{pmatrix}},{\begin{pmatrix}-1\\0\\1\end{pmatrix}}\right\rangle }](http://178.128.105.246/host-https-wikimedia.org/api/rest_v1/media/math/render/svg/81437217e7f1ffccc113bebe4476411d8c059579)
is a basis for the space.
The next result verifies that
the process used in those examples works with any basis for any
subspace of an
(we are restricted to
only because we have not given a
definition of orthogonality for other vector spaces).
- Proof
We will use induction to check that each
is nonzero,
is in the span of
and is orthogonal to all preceding vectors:
.
With those, and with
Corollary 2.3, we will have that
is a basis for the same space as
.
We shall cover the cases up to
, which give the
sense of the argument.
Completing the details is Problem 15.
The
case is trivial— setting
equal to
makes it a nonzero vector since
is a member of a basis,
it is obviously in the desired span,
and the "orthogonal to all preceding vectors" condition is vacuously met.
For the
case, expand the definition of
.
![{\displaystyle {\vec {\kappa }}_{2}={\vec {\beta }}_{2}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {\beta }}_{2}})={\vec {\beta }}_{2}-{\frac {{\vec {\beta }}_{2}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\kappa }}_{1}={\vec {\beta }}_{2}-{\frac {{\vec {\beta }}_{2}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\beta }}_{1}}](http://178.128.105.246/host-https-wikimedia.org/api/rest_v1/media/math/render/svg/104528b528834b806090892ff62d5ff8d34e848f)
This expansion shows that
is nonzero or else this
would be a non-trivial linear dependence among the
's
(it is nontrivial because the coefficient of
is
)
and also shows that
is in the desired span.
Finally,
is orthogonal to the only preceding vector
![{\displaystyle {\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{2}={\vec {\kappa }}_{1}\cdot ({\vec {\beta }}_{2}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {\beta }}_{2}}))=0}](http://178.128.105.246/host-https-wikimedia.org/api/rest_v1/media/math/render/svg/3cff7a28fdd5309be9b0afec02f14c5addae41dc)
because this projection is orthogonal.
The
case is the same as the
case except for one detail.
As in the
case, expanding the definition
![{\displaystyle {\begin{array}{rl}{\vec {\kappa }}_{3}&={\vec {\beta }}_{3}-{\frac {{\vec {\beta }}_{3}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\kappa }}_{1}-{\frac {{\vec {\beta }}_{3}\cdot {\vec {\kappa }}_{2}}{{\vec {\kappa }}_{2}\cdot {\vec {\kappa }}_{2}}}\cdot {\vec {\kappa }}_{2}\\&={\vec {\beta }}_{3}-{\frac {{\vec {\beta }}_{3}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\beta }}_{1}-{\frac {{\vec {\beta }}_{3}\cdot {\vec {\kappa }}_{2}}{{\vec {\kappa }}_{2}\cdot {\vec {\kappa }}_{2}}}\cdot {\bigl (}{\vec {\beta }}_{2}-{\frac {{\vec {\beta }}_{2}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\beta }}_{1}{\bigr )}\end{array}}}](http://178.128.105.246/host-https-wikimedia.org/api/rest_v1/media/math/render/svg/5432eaca9780bd9486b19bbf3e265aa67f829b7c)
shows that
is nonzero and is in the span.
A calculation shows that
is orthogonal to the preceding vector
.
![{\displaystyle {\begin{array}{rl}{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{3}&={\vec {\kappa }}_{1}\cdot {\bigl (}{\vec {\beta }}_{3}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {\beta }}_{3}})-{\mbox{proj}}_{[{\vec {\kappa }}_{2}]}({{\vec {\beta }}_{3}}){\bigr )}\\&={\vec {\kappa }}_{1}\cdot {\bigl (}{\vec {\beta }}_{3}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {\beta }}_{3}}){\bigr )}-{\vec {\kappa }}_{1}\cdot {\mbox{proj}}_{[{\vec {\kappa }}_{2}]}({{\vec {\beta }}_{3}})\\&=0\end{array}}}](http://178.128.105.246/host-https-wikimedia.org/api/rest_v1/media/math/render/svg/6fa72b070f98ecf2d0c3c8d3101ec1e34844902c)
(Here's the difference from the
case— the second line has
two kinds of terms.
The first term is zero because this projection is orthogonal, as in the
case.
The second term is zero because
is orthogonal
to
and so is orthogonal to any vector
in the line spanned by
.)
The check that
is also
orthogonal to the other preceding vector
is similar.
Beyond having the vectors in the basis be orthogonal, we can do more; we can arrange for each vector to have length one by dividing each by its own length (we can normalize the lengths).
- Example 2.8
Normalizing the length of each vector in the orthogonal basis of
Example 2.6
produces this orthonormal basis.
![{\displaystyle \left\langle {\begin{pmatrix}1/{\sqrt {3}}\\1/{\sqrt {3}}\\1/{\sqrt {3}}\end{pmatrix}},{\begin{pmatrix}-1/{\sqrt {6}}\\2/{\sqrt {6}}\\-1/{\sqrt {6}}\end{pmatrix}},{\begin{pmatrix}-1/{\sqrt {2}}\\0\\1/{\sqrt {2}}\end{pmatrix}}\right\rangle }](http://178.128.105.246/host-https-wikimedia.org/api/rest_v1/media/math/render/svg/7f2aa2652e66e2f683cb32f726d9586582a3532c)
Besides its intuitive appeal, and its analogy with the
standard basis
for
, an orthonormal basis also simplifies
some computations.
See Exercise 9, for example.
- This exercise is recommended for all readers.
- Problem 2
Perform the Gram-Schmidt process on each of these bases
for
.
-
-
Then turn those orthogonal bases into orthonormal bases.
- This exercise is recommended for all readers.
- Problem 3
Find an orthonormal basis for this subspace of
: the
plane
.
- Problem 4
Find an orthonormal basis for this subspace of
.
![{\displaystyle \{{\begin{pmatrix}x\\y\\z\\w\end{pmatrix}}\,{\big |}\,x-y-z+w=0{\text{ and }}x+z=0\}}](http://178.128.105.246/host-https-wikimedia.org/api/rest_v1/media/math/render/svg/c76545bdf3f06eb6c869f9e492d533463651086a)
- Problem 5
Show that any linearly independent subset of
can be
orthogonalized without changing its span.
- This exercise is recommended for all readers.
- Problem 6
What happens if we apply the Gram-Schmidt process to
a basis that is already orthogonal?
- Problem 7
Let
be a set of mutually orthogonal vectors in
.
- Prove that for any
in the space, the vector
is orthogonal to each of
, ...,
.
- Illustrate the prior item in
by using
as
, using
as
, and
taking
to have components
,
, and
.
- Show that
is the vector in the
span of the set of
's that is closest to
.
Hint. To the illustration done for the prior part,
add a vector
and apply the Pythagorean Theorem to the resulting triangle.
- Problem 8
Find a vector in
that is orthogonal to both of these.
![{\displaystyle {\begin{pmatrix}1\\5\\-1\end{pmatrix}}\qquad {\begin{pmatrix}2\\2\\0\end{pmatrix}}}](http://178.128.105.246/host-https-wikimedia.org/api/rest_v1/media/math/render/svg/9ab9430455053d330d535b2c0ddb08a780b53293)
- This exercise is recommended for all readers.
- Problem 9
One advantage of orthogonal bases is that they simplify finding the
representation of a vector with respect to that basis.
- For this vector and this non-orthogonal basis for
![{\displaystyle {\vec {v}}={\begin{pmatrix}2\\3\end{pmatrix}}\qquad B=\left\langle {\begin{pmatrix}1\\1\end{pmatrix}},{\begin{pmatrix}1\\0\end{pmatrix}}\right\rangle }](http://178.128.105.246/host-https-wikimedia.org/api/rest_v1/media/math/render/svg/27e4d554d904c3c934f4ad1e85db97f003c895af)
first represent the vector with respect to the basis.
Then project the vector onto the span of each basis vector
and
.
- With this orthogonal basis for
![{\displaystyle K=\left\langle {\begin{pmatrix}1\\1\end{pmatrix}},{\begin{pmatrix}1\\-1\end{pmatrix}}\right\rangle }](http://178.128.105.246/host-https-wikimedia.org/api/rest_v1/media/math/render/svg/5df89d3d8eb3eb0dc43b5de73869af48e6eaf6ba)
represent the same vector
with respect to the basis.
Then project the vector onto the span of each basis vector.
Note that the coefficients in the representation and the projection
are the same.
- Let
be an orthogonal basis for some subspace of
.
Prove that for any
in the subspace,
the
-th component of the representation
is the scalar coefficient
from
.
- Prove that
.
- Problem 10
Bessel's Inequality.
Consider these orthonormal sets
![{\displaystyle B_{1}=\{{\vec {e}}_{1}\}\quad B_{2}=\{{\vec {e}}_{1},{\vec {e}}_{2}\}\quad B_{3}=\{{\vec {e}}_{1},{\vec {e}}_{2},{\vec {e}}_{3}\}\quad B_{4}=\{{\vec {e}}_{1},{\vec {e}}_{2},{\vec {e}}_{3},{\vec {e}}_{4}\}}](http://178.128.105.246/host-https-wikimedia.org/api/rest_v1/media/math/render/svg/d7d7171d03162adb365caa1be2d9519f571c63c3)
along with the vector
whose components are
,
,
, and
.
-
Find the coefficient
for the projection of
onto the span of the vector in
.
Check that
.
-
Find the coefficients
and
for the projection of
onto the spans of the two vectors in
.
Check that
.
- Find
,
, and
associated with the vectors in
, and
,
,
, and
for the vectors in
.
Check that
and
that
.
Show that this holds in general: where
is an orthonormal set and
is coefficient of
the projection of a vector
from the space
then
.
Hint. One way is to look at the inequality
and expand the
's.
- Problem 11
Prove or disprove: every vector in
is in some orthogonal
basis.
- Problem 12
Show that the columns of an
matrix form an orthonormal
set if and only if the inverse of the matrix is its transpose.
Produce such a matrix.
- Problem 13
Does the proof of Theorem 2.2 fail to consider the
possibility that the set of vectors is empty (i.e., that
)?
- Problem 14
Theorem 2.7 describes a change of basis
from any basis
to one that is orthogonal
.
Consider the change of basis matrix
.
- Prove that the matrix
changing bases in the direction opposite to that of the theorem
has an upper triangular shape— all
of its entries below the main diagonal are zeros.
- Prove that the inverse of an upper triangular matrix is
also upper triangular (if the matrix is invertible, that is).
This shows that the matrix
changing bases
in the direction described in the theorem is upper triangular.
- Problem 15
Complete the induction argument in the proof of
Theorem 2.7.
Solutions