0

Consider a chain with state space $\{1,2, \cdots \}.$ If you are at $1$ go to state $j$ with probability $p_j$ $($$ j=1,2,\cdots$ $) ,$ where these are non-negative numbers adding to $1$. If you are in a state $i>1\ ,$ then go just one step back$,$ that is$,$ to $i-1.$ Discuss the nature of the states and nature of the stationary distribution$.$

Assuming that all $p_j$'s are positive I found that the Markov chain is irreducible with all it's states recurrent$.$ So there is no transient state and hence inessential state in the above Markov chain$.$ While calculating stationary distribution I found a problem$.$ Here it is $:$

Suppose $\pi = (\pi_1 , \pi_2 , \cdots )$ be the stationary distribution of the above Markov chain$.$ Then I found that $\pi_2 = (1-p_1) \pi_1, \pi_3 = (1-p_1-p_2) \pi_1, \cdots$ Since $\sum\limits_{i=1}^{\infty} \pi_i = 1$ so $\pi_1 \{1 + (1-p_1) + (1-p_1 -p_2) + (1-p_1 -p_2 - p_3) + \cdots \}= 1.$ Now how do I find the sum

$$1 + \sum_{k=1}^{\infty} (1-p_1-p_2 -p_3 - \cdots - p_k)\ ?$$

Or in other words

$$\sum\limits_{k=1}^{\infty} \sum\limits_{j=k}^{\infty} p_j\ ?$$

This gives us $\pi_1$ and consequently all the $\pi_i$'s$.$ where

$$\pi_i = \frac {\sum\limits_{k=i}^{\infty} p_k} {\sum\limits_{k=1}^{\infty} \sum\limits_{j=k}^{\infty} p_j}.$$

for $i=1,2,\cdots.$

Please help me in this regard. Thank you very much.

little o
  • 4,853

1 Answers1

1

$$\begin{align} \sum_{j=1}^\infty{jp_j}&=\sum_{j=1}^\infty{p_j}+\sum_{j=2}^\infty{(j-1)p_j}\\ &=1+\sum_{j=2}^\infty{p_j}+\sum_{j=3}^\infty{(j-2)p_j}\\ &=1+(1-p_1)+\sum_{j=3}^\infty{p_j}+\sum_{j=4}^\infty{(j-3)p_j}\\ &\vdots \end{align}$$

EDIT

This is true if the given sum converges. As discussed in the accepted answer to this question, it need not.

saulspatz
  • 53,131
  • I have understood it already. Thanks for your answer. – little o Dec 05 '18 at 15:37
  • @Dbchatto67 Right, I'd just finished typing it when I saw your comment, so I went ahead and posted it. An interesting point is that this problem shows if $p_j\ge0$ and $\sum{p_j}=1,$ then $\sum{jp_j}$ converges. Is there a more direct proof? – saulspatz Dec 05 '18 at 15:44
  • Hmm. Good observation. May be there are direct approaches but I am not familiar with those. Sorry for that. Do you know some of them? Then please don't forget to share your thoughts. That will also be helpful for me. – little o Dec 05 '18 at 15:54
  • @Dbchatto67 There must be an error here. Look at https://math.stackexchange.com/questions/3027282/simple-proof-of-a-theorem-on-convergence-of-series/3027293#3027293 – saulspatz Dec 05 '18 at 16:30
  • Then what is the error lying in the question? – little o Dec 05 '18 at 16:36
  • Then $\pi_i$'s are $\frac {\infty} {\infty}$ form. Isn't it so? – little o Dec 05 '18 at 16:38
  • You have done the right job in posting it in MSE as it is doubtful. But I am surprised that nobody ventured to talk about this linked question. – little o Dec 05 '18 at 16:42
  • @saulspatz Looked it up. Didn't find any mistake. Rather, for your answer to hold, you must show that $$\lim_{k\to +\infty}\sum_{j=k}^\infty(j-k)p_j=0,$$ which is not obvious to me. – Scientifica Dec 05 '18 at 17:12
  • @Scientifica Hmmmm. I need to think about this. Thanks. – saulspatz Dec 05 '18 at 17:45
  • It's a pleasure! I tried to find a counter-example but I couldn't :/ – Scientifica Dec 05 '18 at 18:02
  • @Scientifica Several are given at https://math.stackexchange.com/questions/3027282/simple-proof-of-a-theorem-on-convergence-of-series?noredirect=1#comment6242166_3027282 – saulspatz Dec 05 '18 at 18:05
  • Thank you. I'm actually one of the answerers there xD (in fact the first to answer :p) What I meant is, I don't know if there's a counter-example for $\sum\sum p_i$ converges but $\sum jp_j$ diverges. – Scientifica Dec 05 '18 at 18:10
  • @Scientifica Let $p_{ij}=p_j,/ j\ge i,$ and $0,$ otherwise. The double sum is the sum of the rows, and the sum I gave is the sum of the columns, so there are simultaneously convergent, and equal when they converge, by Tonelli's theorem. – saulspatz Dec 05 '18 at 18:18