"Ignoring issues" is never good advice.
An obvious difference you have with the finite-dimensional case is that in finite dimension you can take any coefficients $c_1,\ldots,c_n$ and form $\sum_jc_je_j$. In an infinite-dimensional Hilbert space you need to make sure that $\sum_j|c_j|^2<\infty$ to be able to consider $\sum_jc_je_j$.
Regarding bounded operators, a drastic example is given by the trace. Given an orthonormal basis $\{e_n\}$ one can write, as in the finite-dimensional case,
$$\tag1
\operatorname{Tr}(A)=\sum_n\langle Ae_n,e_n\rangle.
$$
But this fails to make sense for most operators. The operators $A$ for which this makes sense are called the trace-class operators and they are a subclass of the compact operators. Here's an example of how things can go wrong when $A$ is not trace-class. Let $A$ be the selfadjoint compact operator
$$
A=\bigoplus_n\begin{bmatrix} 0& 1/n\\ 1/n&0\end{bmatrix}.
$$
This is the same as saying that, for some fixed orthonormal basis $\{e_n\}$,
$$
Ae_{2n}=\frac{1}n\,e_{2n-1},\qquad Ae_{2n-1}=\frac 1n\,e_{2n}.
$$
The naive computation in $(1)$ would give $\operatorname{Tr}(A)=0$. But the property that makes the trace interesting is that $\operatorname{Tr}(AB)=\operatorname{Tr}(BA)$, which is equivalent to say that $(1)$ does not depend on the choice of the orthonormal basis. If we use the orthonormal basis
$$
\frac{e_1+e_2}{\sqrt2},\frac{e_1-e_2}{\sqrt2},
\frac{e_3+e_4}{\sqrt2},\frac{e_3-e_4}{\sqrt2},\ldots
$$
now the diagonal of $A$ is
$$\tag2
1,-1,\frac12,-\frac12,\ldots
$$
This still converges to $0$, so it seems like all is good. Until we realize that any permutation of an orthonormal basis is an orthonormal basis, and that the series converges conditionally. So the diagonal of $A$ can be made any permutation of the sequence in $(2)$, which means that for each $r\in\mathbb R\cup\{\infty,-\infty\}$ there exists an orthonormal basis such that $(1)$ gives $\operatorname{Tr}(A)=r$.