1

Let the transition matrix for a stochastic process be

$$P=\begin{bmatrix} 1 & 0.7 & 0 & 0 & 0 \\ 0 & 0 & 0.5 & 0 & 0 \\ 0 & 0.3 & 0 & 0.65 & 0 \\ 0 & 0 & 0.5 & 0 & 0 \\ 0 & 0 & 0 & 0.35 & 1 \end{bmatrix}$$

and let $\mathbf{x}_0$ be the initial state. Let $s_1$ be the state with the vector

$$\mathbf{x}_0=\begin{bmatrix}1 \\ 0 \\ 0 \\ 0 \\ 0 \end{bmatrix}$$

The communication class (I don't know the English word) $K_1 = \{s_1\}$ is a closed class, with means that if $s_1 \leadsto s_i$ then $s_i \in K_1$.

The formula for calculating the probabilty that an initial state $s_j$ will end up in $K_1$, sooner or later, is

$$x_j^{K_1} = 1, s_j \in K_1$$ $$x_j^{K_1} = \sum_{i=1}^n{p_{ij}x_i}, s_j \notin K_1$$

where $p_{ij}$ is the entry in row $i$ and column $j$ in P.

Here comes my problem. I get the following result: if the initial state is $s_2$ then the probability that the process will end in $K_1$ is $0$!

Because I solve the system $P\mathbf{x} = I_5\mathbf{x}$ which is $(P-I_5)=\mathbf{0}$. The solution is the vector

$$\begin{bmatrix}x_1 \\ 0 \\ 0 \\0 \\ x_5 \end{bmatrix}$$

which is

$$\begin{bmatrix}1 \\ 0 \\ 0 \\0 \\ 0 \end{bmatrix}$$

in relation to $K_1$.

This I cannot understand. The probability that $s_2$ could end up in $K_1$ can't be zero?

Ganesh
  • 1,721
MathMag
  • 21
  • Is $s_2$ an arbitrary state? Also, what does it mean $x_j^{K_1}$? – MASL Sep 20 '15 at 11:51
  • 1
    The system Px=Ix does not yield the hitting probabilities you are after but the stationary distributions (which, here, are all the distributions supported by {1,5}). What is your source for this? – Did Sep 20 '15 at 11:55
  • $s_2$ is the initial state x = [0;1;0;0;0]. $x_j^{K_1}$ means the probability that $s_2$ sooner or later will end up in $K_1$ – MathMag Sep 20 '15 at 11:59
  • Did: I don't have a specific source, I just try to understand this... – MathMag Sep 20 '15 at 12:00
  • The probabilities $h_i$ to hit $1$ starting from $i$ solve the system $h_2=.7+.3h_3$, $h_3=.5h_2+.5h_4$, $h_4=.65h_3$. You are asking for $h_2$. – Did Sep 20 '15 at 12:01
  • Did: thanks, I didn't think it was that simple. There was a previous exercise that involved $P-I_5$ so I just thought it was related... but thanks a lot, I can of course solve that system. – MathMag Sep 20 '15 at 12:08

0 Answers0