Intuition on Changing Order of Summations

Question

$$ \sum_{i=1}^{\infty}\left[\sum_{j=i}^{\infty}f(i,j)\right]=\sum_{j=1}^{\infty}\left[\sum_{i=1}^{j}f(i,j)\right] $$

I have trouble understanding the change of order of this summation. I know that $$ 1≤i≤j≤\infty $$ however, I can't seem to make sense of this even with manually comparing each sum of both double summations.

---

In the left side, we always have the sums: $$ S_1=f(1,1)+\dots f(1,\infty) $$ $$ S_2=f(2,2)+\dots f(2,\infty) $$ $$ \vdots $$ so the total sum

$$ S=\sum S_n $$ looks like summing each element of an $N\text{x}N$ matrix as $N\to\infty$

---

In the right side, we have the sums:

$$ S_1=f(1,1) $$ $$ S_2=f(1,2) + f(2,2) $$ $$ S_3=f(1,3)+f(2,3) + f(3,3) $$ $$ \vdots $$ The total sum $S$ looks like summing the top-right to bottom-left entries of a "pyramid" structure, which looks like the earlier matrix but deformed and slanted. However, I do not want to just assume that this is the case since I am working with infinite sums.

Is there a better intuition on how changing order of summations work so that I may extend it to more difficult scenarios particularly in the proof of the convolution property of Z transforms.

Answer 1

Intuition:
The change of summation work much like you are trying to loop over some $N\times N$ matrix (in your case you take $N\to\infty$). You can go by looking at each rows and within each columns or change it and go by looking at columns and each rows.

In your example lets say $i$ is the row and $j$ is the column index.

The right hand side:
$\sum_{i=1}^{\infty}$ means that you are looking through all row and $\sum_{j=i}^{\infty}$ means that you are looking for elements below the diagonal(row is already set by $i$ so the column identifies the element $(i, j)$).

The left hand side:
$\sum_{j=1}^{\infty}$ means that you are looking through all columns and $\sum_{i=1}^{j}$ means again that you are (again) looking for elements below the diagonal(now the column is already set by $j$ so the row identifies the element $(i, j)$).

You could prove that it works for any $N\times N$ matrix by induction.

Answer 2

The inequality chain \begin{align*} \color{blue}{1\leq i\leq j<\infty}\tag{1} \end{align*} is already a key to gain some insight for the series transformations. Here we do an index transformation in small steps which might be helpful to better see what's going on.

Assuming the double series is absolutely convergent, so that we can reorder terms at will, we obtain \begin{align*} \color{blue}{\sum_{i=1}^\infty\sum_{j=i}^\infty f(i,j)} &=\sum_{1\leq i<\infty}\sum_{i\leq j<\infty}f(i,j)\tag{2.1}\\ &=\sum_{\color{blue}{1\leq i\leq j<\infty}}f(i,j)\tag{2.2}\\ &=\sum_{1\leq j<\infty}\sum_{1\leq i\leq j}f(i,j)\tag{2.3}\\ &\,\,\color{blue}{=\sum_{j=1}^\infty\sum_{i=1}^jf(i,j)} \end{align*}

Comment:

• In (2.1) and (2.3) we just use another common notation without rearranging anything.

• In (2.2) we write the index regions more compactly as one inequality chain, but without changing the index region. This notation implies a rearrangement of the summands.

Hint: Note that $i,j$ in $f(i,j)$ are positive integers. So, $f(1,\infty)$ is not an admissible representation. We rather write \begin{align*} S_1=f(1,1)+f(1,2)+f(1,3)+\cdots \end{align*} to represent $S_1=\sum_{j=1}^\infty f(1,j)$. For the same reasons we also write $1\leq j\color{blue}{<\infty}$.