Orthonormal basis in Hilbert space - 2 questions

Question

I know there have been a number of questions on Hilbert spaces and orthonormal basis, but I can't find any answers to these two questions:

1) Let $H$ be a Hilbert space, and say we found a Hilbert basis by taking a maximal orthonormal set. This question is only interesting if we assume the set of basis vectors is uncountable, so let's do that. I know how to show that the finite linear span of this set is dense in H. But does this imply that every vector in H is a (countably) infinite linear combination of our basis vectors? I know we can approach arbitrarily close using finite linear combinations, but the basis vectors being used in the finite linear combinations may keep changing.

2) My textbook writes expressions such as $\sum_{k=1}^{\infty}\langle v_k,h\rangle v_k$ where $v_k$ forms an orthonormal set in $H$. My question here is: How can we say that the order of the summation does not matter? I know that Bessel's Inequality says $\sum_{k=1}^{\infty}\langle v_k,h\rangle^2$ converges, but I not sure if this helps, or how it helps.

Answer 1

1) The sum of over an uncountable set of indices is defined precisely as the limit over the net of finite sets, ordered by inclusion.

2) Order does not matter because the convergence is absolute. You have$$ \left\|\sum_{k>N}^M\langle v_k,h\rangle\,v_k\right\|^2=\sum_{k=N+1}^M|\langle v_k,h\rangle| $$ and so Bessel's inequality, as you mention, implies that order does not matter, with the exact same proof as the one that absolute convergence implies that order does not matter for sequences of numbers.

Your two questions are related, because defining series as in 1), in the countable case, is precisely equivalent to absolute convergence.

Answer 2

For the first one, the answer is yes. In metric spaces (more generally, first countable spaces), in particular in Hilbert spaces, the closure of a set is just the set of all limits of sequences.

As you have said, the set of finite combinations of basic vectors is dense, so any element of the space is limit of finite combinations; and it's not hard to see that the limit of a sequence of finite combinations is a countable combination.

For the second one: the order of summation doesn't matter because the norm of the tail tends to zero: $\Sigma_{k=m}^{\infty}\lvert\langle v_k,h\rangle\rvert^2 \to 0$.

So if you take any $\varepsilon>0$ and some reordering $w_k$ of $v_k$, then you can find an $N$ such that $\sum_{k=N}^{\infty}\lvert \langle v_k,h\rangle \rvert^2<\varepsilon$ and an $M$ such that $v_1,\ldots,v_{N-1}$ all appear among $w_1,\ldots,w_{M-1}$, so that by triangle inequality: $$\left \lVert \sum_{k=1}^{\infty}\langle v_k,h\rangle v_k- \sum_{k=1}^{\infty}\langle w_k,h\rangle w_k\right\rVert^2\leq \left \lVert\sum_{k=1}^{N-1}\langle v_k,h\rangle v_k-\sum_{k=1}^{N-1}\langle v_k,h\rangle v_k\right \rVert^2 +2 \sum_{k=N}^{\infty}\lvert \langle v_k,h\rangle \rvert^2<2\varepsilon$$

Answer 3

I have been struggling with the same questions when I was studying Hilbert Spaces first time. But I found following answers.

1) every vector in the Hilbert space is NOT a (infinite) combination of the basis vectors (I mean, what does sum of infinite number of "vectors" mean in the first place?). What we have is an isometrical isomorphism between the Hilbert space and $\ell^2(A)$, where $A$ is the index set of our basis. So for every vector $h$ in the Hilbert space, you can find a unique corresponding element $\hat h$ in $\ell^2(A)$, where for every $k \in A$ we have $\hat h(k)=\langle h,v_k\rangle$ (and vice versa). Also, since the mapping is isometry, we have $\lVert h\rVert=\sum_{\alpha \in A}{\langle h,v_k\rangle^2}$. Note that if A is uncountable, $\hat h$ has values on uncountable points.

2) Again I don't think that the "equality" is correct in $h=\sum_{k=1}^{\infty}{\langle h,v_k\rangle v_k}$. Cause as you mentioned, order of summation, convergence and etc are not clear in that summation (Again, I don't know what does an infinite sum of vectors mean). Unless we are trying to give a representation in this form, for the mapping I mentioned in 1).

Answer 4

Thanks to those who replied. I have gathered bits from the answers and decided to write my own answer to better understand the solutions.

1) Fix $h\in H$. First, I claim that there can only be countably many $v_k$ in our orthonormal basis set $S$ for which $\langle v_k,h\rangle \neq0$. Proof: Consider the following partition of the non-negative reals: ${0}, (1,\infty), (1/2,1], (1/3,1/2],(1/4,1/3],...$. If any of the intervals have infinitely many $v_k$ for which $\langle v_k,h\rangle $ belongs to that interval, then we use those $v_k$ and get a contradiction to Bessel's inequality. Since a countable union of finite sets is countable, we have proven our claim.

Given our claim, the sum $\Sigma_{v_k\in S}\langle v_k,h\rangle v_k$ makes sense, being a countable sum (and order does not matter by part 2). It is an easy exercise to see that this sum converges to some $h'$ in $H$. Moreover, $h-h'$ is orthogonal to all $v_k\in S$ (continuity of innrer product is used here). By maximality of $S$, $h=h'$, so $h$ has been exhibited as a countable sum as desired.

2) As tomasz suggested, find $N$ such that $\Sigma_{k=N+1}^{\infty}\langle v_k,h\rangle ^2<\epsilon$, and $M$ such that $v_1,v_2,...,v_N$ appears among $w_1,...,w_M$. Then $\Sigma_{k=1}^{M}\langle v_k,h\rangle v_k-\Sigma_{k=1}^{M}\langle w_k,h\rangle w_k$ has squared norm that is at most $2\epsilon$. So the two infinite sum converge to the same thing.