0

I am electrical engineer and not so much deep into math. I have one question that i think some of you may enlighten me up. The concept Hilbert space; I know the idea is related to Fourier series/transform in signal analysis (etc. The issue of convergence of both series/transform, energy, Parsaval). However with it comes to mathematical context, we have Cauchy sequence, completeness,inner product space, etc.
I would like any of you to share their knowledge on Hilbert space that connects to the signal analysis in the most comprehensive, intuitive and simple way. Thanks

fery
  • 94
  • This can be usefull http://www.fourierandwavelets.org/FSP_v1.1_2014.pdf – Emilio Novati Sep 19 '15 at 21:33
  • @ Emilio Novat: I have got this book and seen the formal explanations.I need a better intuitions not a dried rigorous text book explanation.I am aware of many books.Thanks anyway. – fery Sep 19 '15 at 21:42
  • I am also an electrical engineer and I know a little about their connection, but what exactly do you wish to know? But let me warn you, whatever that comes with understanding Hilbert space will only give support to mathematical formulas you use in signal processing, but will not whatsoever simplify the math or magically calculates you all the fourier coefficients. In fact a good knowledge of linear algebra is enough – Fraïssé Sep 19 '15 at 21:47
  • @ Illegal Immigrant: Plz share your understanding of Hilbert space. I know that if we dont have Hilbert space, basically we can't mathematically construct the Fourier formula. My question is a kind of philosophical one. – fery Sep 19 '15 at 21:52
  • A Hilbert space is a complete inner product space, (please completely ignore what complete means). What it means that you have a vector space, which is the space that satisfies the vector space axioms, and you then define a inner product function on it, making it an inner product space. The inner product function is $\int v^w dt$, is conjugate. Signals are vectors in a vector space, when the vector space has an inner product function $\int v^*w dt$ it becomes a inner product space, which is equivalent to a Hilbert space. Fourier transform (say the continuous one) is an inner product. – Fraïssé Sep 19 '15 at 21:58
  • Now to an engineer it may be confusing as to why signals are vectors, because signals aren't little arrows...they are sinusoids, and why anybody cares about the distinction between a space with an inner product and without an inner product. An engineer will also be confused as to why inner product is $\int v^*w dt$ instead of $x^Ty$ or $|x||y|\cos(\theta)$. Finally, an engineer may wonder about the implication that fourier transform is the inner product. Maybe those questions will give you some directions in your search for their connections. – Fraïssé Sep 19 '15 at 22:04
  • @ Illegal Immigrant: Well your explanation brings up more interesting question mentioned above: why anybody cares about the distinction between a space with an inner product and without an inner product? Also let me know what you really understand from completeness. – fery Sep 19 '15 at 22:15
  • @ Illegal Immigrant: If possible what is your view on the interesting but deep questions you asked that one engineer may confuse? – fery Sep 19 '15 at 22:17
  • To explain completeness you also have to introduce the notion of a normed space. But in simple terms, vector spaces are contained in normed spaces are contained in inner product spaces whereas the opposite is not true. A hilbert space is a normed space, and it is complete (by definition) since all cauchy sequences $(v_n)$ on hilbert space converges. Meaning that all and any sequence where any two elements are close to each other will converge to some value inside of the space. It is a good property, because $(\cos(nt), \sin(nt)$ is complete, which means you are able to do a fourier series – Fraïssé Sep 19 '15 at 22:27

1 Answers1

1

There are two important aspects to a Hilbert space: the inner product and the completeness property. I don't have much of a background in analysis so I can't explain very much about completeness, but I can tell you why the inner product is important for the Fourier transform.

The inner product is important because it gives us the concept of orthogonality, and orthogonality is import because having an orthonormal basis makes it very easy to break a vector up into its components in that basis.

Let's take the example of a vector $\textbf{v}$ in $\mathbb{R}^3$ with an orthonormal basis $\{\textbf{e}_1,\textbf{e}_2,\textbf{e}_3 \}$. Let's see how we get its first component $v_1$ (recall that with an orthonormal basis, $(\textbf{e}_i \cdot \textbf{e}_i) = 1$ and $(\textbf{e}_i \cdot \textbf{e}_{j \neq i}) = 0$):

$$\textbf{v} = v_1 \textbf{e}_1 + v_2 \textbf{e}_2 + v_3 \textbf{e}_3$$ $$\textbf{e}_1 \cdot \textbf{v} = \textbf{e}_1 \cdot (v_1 \textbf{e}_1 + v_2 \textbf{e}_2 + v_3 \textbf{e}_3)$$ $$\textbf{e}_1 \cdot \textbf{v} = v_1 (\textbf{e}_1 \cdot \textbf{e}_1) + v_2 (\textbf{e}_1 \cdot \textbf{e}_2) + v_3 (\textbf{e}_1 \cdot \textbf{e}_3)$$

$$\textbf{e}_1 \cdot \textbf{v} = v_1$$

Because many of the inner products go to zero, we arrive at the very simple formula $v_i = \textbf{e}_i \cdot \textbf{v}$.

In the case of the space of square-integrable functions, we have the inner product on two functions $\langle f,g \rangle = \int f(x)g(x) dx$. This means we understand what it means for functions to be orthogonal.

It happens to be that the set $\{ cos(\frac{2\pi t}{T} n), sin(\frac{2\pi t}{T} n) \}_{n=0}^{\infty} $ is orthogonal over an interval of length $T$. If we have some generic function $f(t)$ defined on that interval, we can just to express it as a linear combination of basis functions in this set:

$$f(t) = \sum_{n=0}^{\infty} a_n cos(\frac{2\pi t}{T} n) + \sum_{m=1}^{\infty} b_m sin(\frac{2\pi t}{T} m)$$

Let's try using the inner product to get the coefficients $a_i$ by taking the inner product with $cos(\frac{2\pi t}{T} i)$:

$$\langle cos(\frac{2\pi t}{T} i), f(t) \rangle = \left\langle cos(\frac{2\pi t}{T} i), \sum_{n=0}^{\infty} a_n cos(\frac{2\pi t}{T} n) + \sum_{m=1}^{\infty} b_m sin(\frac{2\pi t}{T} m) \right\rangle$$

$$\langle cos(\frac{2\pi t}{T} i), f(t) \rangle = \sum_{n=0}^{\infty} a_n \left\langle cos(\frac{2\pi t}{T} i), cos(\frac{2\pi t}{T} n) \right\rangle + \sum_{m=1}^{\infty} b_m \left\langle cos(\frac{2\pi t}{T} i),sin(\frac{2\pi t}{T} m) \right\rangle$$

$$\langle cos(\frac{2\pi t}{T} i), f(t) \rangle = a_i \langle cos(\frac{2\pi t}{T} i), cos(\frac{2\pi t}{T} i) \rangle $$

$$\int_{t=0}^{T} cos(\frac{2\pi t}{T} i) f(t) dt = a_i \int_{t=0}^{T} cos^2(\frac{2\pi t}{T} i)dt $$

$$\int_{t=0}^{T} cos(\frac{2\pi t}{T} i) f(t) dt = a_i \frac{T}{2} $$

$$a_i = \frac{2}{T} \int_{t=0}^{T} cos(\frac{2\pi t}{T} i) f(t) dt$$

Which is the familiar formula for the Fourier cosine coefficients. A similar procedure can be used to derive the sine coefficients $b_m$.

Now, we haven't show why $\{ cos(\frac{2\pi t}{T} n), sin(\frac{2\pi t}{T} n) \}_{n=0}^{\infty}$ is in fact a complete basis, or that the infinite sum in the equation above actually converges (this is beyond my knowledge), but I know this can be shown using the fact that Hilbert spaces are complete.

eigenchris
  • 2,112
  • 12
  • 22
  • Great way of demonstrating the vector space perspective on signal processing. With most texts I've seen on the topic, it would have taken weeks to arrive at these simple conclusions, and even then you may not recognize that it happened. – orodbhen Jan 27 '18 at 10:29