15.14: Approximation and Projections in Hilbert Space
- Page ID
- 23204
Introduction
Given a line 'l' and a point 'p' in the plane, what's the closest point 'm' to 'p' on 'l'?
Same problem: Let \(x\) and \(v\) be vectors in \(\mathbb{R}^2\). Say \(\|v\|=1\). For what value of \(\alpha\) is \(\|x-\alpha v\|_{2}\) minimized? (what point in span{v} best approximates \(x\)?)
The condition is that \(x-\widehat{\alpha} v\) and \(\alpha v\) are orthogonal.
Calculating α
How to calculate \(\widehat{a}\)?
We know that (\(x-\widehat{\alpha} v\)) is perpendicular to every vector in span{v}, so
\[\begin{array}{l}
\langle x-\widehat{\alpha} v, \beta v\rangle=0, \forall(\beta) \\
\bar{\beta}\langle(x, v)\rangle-\widehat{\alpha} \bar{\beta}\langle(v, v)\rangle=0
\end{array} \nonumber \]
because \(\langle v, v\rangle=1\), so
\[(\langle(x, v)\rangle-\hat{\alpha}=0) \Rightarrow(\hat{\alpha}=\langle x, v\rangle) \nonumber \]
Closest vector in span{v} = \(\langle(x, v)\rangle v\), where \(\langle(x, v)\rangle v\) is the projection of \(x\) onto \(v\).
We can do the same thing in higher dimensions.
Exercise \(\PageIndex{1}\)
Let \(V \subset H\) be a subspace of a Hilbert space (Section 15.4) \(H\). Let \(x \in H\) be given. Find the \(y \in V\) that best approximates \(x\). i.e., \(\|x-y\|\) is minimized.
- Answer
-
- Find an orthonormal basis \(\left\{b_{1}, \ldots, b_{k}\right\}\) for \(V\)
- Project \(x\) onto \(V\) using \[y=\sum_{i=1}^{k}\left\langle\left(x, b_{i}\right)\right\rangle b_{i} \nonumber \] then \(y\) is the closest point in \(V\) to x and \((x-y) \perp V(\langle x-y, v\rangle=0, \quad \forall(v) \in V\)
Example \(\PageIndex{1}\)
\(x \in \mathbb{R}^{3}\), \(V=\operatorname{span}\left(\left\{\left(\begin{array}{l}
1 \\
0 \\
0
\end{array}\right),\left(\begin{array}{l}
0 \\
1 \\
0
\end{array}\right)\right\}\right)\), \(x=\left(\begin{array}{l}
a \\
b \\
c
\end{array}\right)\). So,
\[y=\sum_{i=1}^{2}\left\langle\left(x, b_{i}\right)\right\rangle b_{i}=a\left(\begin{array}{l}
1 \\
0 \\
0
\end{array}\right)+b\left(\begin{array}{l}
0 \\
1 \\
0
\end{array}\right)=\left(\begin{array}{l}
a \\
b \\
0
\end{array}\right) \nonumber \]
Example \(\PageIndex{2}\)
V = {space of periodic signals with frequency no greater than \(3w_0\)}. Given periodic f(t), what is the signal in V that best approximates f?
- \(\left\{\frac{1}{\sqrt{T}} e^{j w_{0} k t}, k=-3,-2, \dots, 2,3\right\}\) is an ONB for V
- \(g(t)=\frac{1}{T} \sum_{k=-3}^{3}\left\langle\left(f(t), e^{j w_{0} k t}\right)\right\rangle e^{j w_{0} k t}\) is the closest signal in V to f(t) ⇒ reconstruct f(t) using only 7 terms of its Fourier series.
Example \(\PageIndex{3}\)
Let V = {functions piecewise constant between the integers}
- ONB for V.
\[b_{i}=\left\{\begin{array}{l}
1 \text { if } i-1 \leq t<i \\
0 \text { otherwise }
\end{array}\right. \nonumber \]
where \(\left\{b_{i}\right\}\) is an ONB.
Best piecewise constant approximation?
\[\begin{array}{c}
g(t)=\sum_{i=-\infty}^{\infty}\left\langle\left(f, b_{i}\right)\right\rangle b_{i} \\
\left\langle f, b_{i}\right\rangle=\int_{-\infty}^{\infty} f(t) b_{i}(t) \mathrm{d} t=\int_{i-1}^{i} f(t) \mathrm{d} t
\end{array} \nonumber \]
Example \(\PageIndex{4}\)
This demonstration explores approximation using a Fourier basis and a Haar Wavelet basis. See here for instructions on how to use the demo.