I have been taking a look at some basic properties of discrete markov chains. Let $S$ be a Markov Chain with a finite state space and with transition matrix $P$. It is well know that under some regularities properties there is an invariant distribution such as $\Pi=P^\prime \Pi$. My question is about the probability $\Pi_i P_{ij}$ related to the transition from $i$ to $j$, i.e., $\Pi_i P_{ij}$ is the probability that the transition $i$ to $j$ happens. Are these probabilities important somewhere? I mean "Do they arise in any important context?
Asked
Active
Viewed 39 times
0
-
Check out the M/M/1 queue for a simple and important application, https://en.m.wikipedia.org/wiki/M/M/1_queue – ericf Oct 16 '18 at 22:49
-
@ericf where does it exactly arise? Sorry, I could not get! – DanielTheRocketMan Oct 16 '18 at 22:56
-
It is used in the subsection entitled stationary analysis. – ericf Oct 16 '18 at 22:57
-
@ericf Sure. You mean $\Pi$. I am realy interested about the product $\Pi_i\times P_{ij}$. – DanielTheRocketMan Oct 16 '18 at 23:00
-
If j=i+1 then that would be the probability that you observe a transition from i customers to i+1 customers. For example in a computer system if you have enough local memory for 100 items then this could be the probability that you would need to transition to external memory for additional items which could incur an additional charge if you are using cloud computing. – ericf Oct 16 '18 at 23:09