1

Consider a gambling game in which on any turn you win \$1 with the probability $p=0.4$ and you loose \$1 with the probability $p=0.4$. We have that $p(i,j)=p\{X_{n+1}=j\mid X_n=i\}$ and thus, for $N=5$,

$$\big(p(i,j)\big)_{0\leq i,j\leq 5}=\begin{pmatrix}1.0&0&0&0&0&0\\0.6&0&0.4&0&0&0\\0&0.6&0&0.4&0&0\\0&0&0.6&0&0.4&0\\0&0&0&0.6&0&0.4\\0&0&0&0&0&1.0\end{pmatrix}.$$

I don't understand why $p(0,0)=1=p(N,N)$. Morevove, I don't see how to read this matrix. Is for example $p(3,4)=p\{X_4=4\mid X_3=3\}$ ?

And is $p(0,0)=p\{X_0=0\mid X_0=0\}$ and $p(5,5)=p\{X_5=5\mid X_5=5\}$. My interpretation looks strange...

EDIT

Since I don't have any answer from Tlön Uqbar Orbis Tertius, let see this example: Let $X_n$ be the wether on the day $n$. The weather it's not exactly a Markov chain, but we'll suppose that it is. Let $1=rainy$ and $2=sunny$. How would you interpret $$\begin{array}{ccc}&\boldsymbol 1&\boldsymbol 2\\ \boldsymbol 1&.6&.4\\\boldsymbol 2&.2&.8\end{array} \ \ \ \ ?$$ Does $p(1,2)=p\{X-2=2\mid X_1=1\}$ ? If yes, how to compute $p(n,n+1)$ if $n>2$ ? Because here we just have the results for $n\in\{1,2\}$.

idm
  • 11,824

3 Answers3

1

When you're ruined, you're definitely ruined. That's what says $p(0,0) = 1$ : if your personnal wealth is zero, you can't gamble anymore, you stay at the state $0$. Now let $N$ be the fortune of the casino. If you earned $N$, then the casino is ruined, and you can't gamble anymore too : that's the meaning of $p(N,N) = 1$. You've earned everything, so the game is done, you stay at the state $N$. These two states are called "absorbing", because once you reach them, you can't get out.

And your interpretation of $p(3,4)$ is the right one : in general, $p(x,y)$ is the probability to go at the state $y$, knowing you're now at the state $x$.

Tlön Uqbar Orbis Tertius
  • 3,194
  • I think this answer needs to pay more attention to OP's usage of the subscripts of $X_{n}$ and the meaning of this random variable. I wouldn't really call this interpretation correct just because the $i$'s and the $j$'s are plugged into the correct spots. – JessicaK May 30 '15 at 12:43
  • Aie aie... I still have some problem to understand (and sorry about that). May be this example. Let see this example: Let $X_n$ be the wether on the day $n$. The weather it's not exactly a Markov chain, but we'll suppose that it is. Let $1=rainy$ and $2=sunny$. How would you interpret $$\begin{array}{ccc} & \boldsymbol 1& \boldsymbol 2\ \boldsymbol 1&.6 &.4\ \boldsymbol 2& .2&.8\end{array} $$ Does $p(1,2)=p{X_2=2\mid X_1=1}$ ? If yes, how to compute $p(n,n+1)$ ? because here $n=2$ ? – idm May 30 '15 at 14:51
  • Yep this should do it, no? :) – mathreadler May 31 '15 at 09:46
1

Let's look at your weather example:

Let $X_n$ be the wether on the day $n$. The weather it's not exactly a Markov chain, but we'll suppose that it is. Let $1=Rainy$ and $2=Sunny$. How would you interpret $$\begin{array}{ccc}&\boldsymbol 1&\boldsymbol 2\\ \boldsymbol 1&.6&.4\\\boldsymbol 2&.2&.8\end{array} \ \ \ \ ?$$ Does $p(1,2)=p\{X_2=2\mid X_1=1\}$ ? If yes, how to compute $p(n,n+1)$ if $n>2$ ? Because here we just have the results for $n\in\{1,2\}$.

My Explanation:

This means that if you are in state 1 ($Rainy$) at time $n$, there is a 0.6 probability you will be at state 1 ($Rainy$) at time $n + 1$ and a 0.4 probability that you will be at state 2 ($Sunny$) at time $n + 1$.

A Discrete Time Markov Chain describes the evolution of a system (defined by a state space $S = \{s_1, s_2, ..., s_N\}$) over discrete time periods $0, 1, 2, ..., T$.

$X_n$ is the state that your system is in (in your example it is either 1 ($Rainy$) or 2 ($Sunny$)) at time $n$.

Helpful Information Related to Your Question:

$p\{X_2=1\mid X_1=1\}$ = $p\{X_3=1\mid X_2=1\}$ = $p\{X_4=1\mid X_3=1\}$ = P(weather goes from $Rainy$ to $Rainy$ in 1 step) = 0.6

$p\{X_2=2\mid X_1=1\}$ = $p\{X_3=2\mid X_2=1\}$ = $p\{X_4=2\mid X_3=1\}$ = P(weather goes from $Rainy$ to $Sunny$ in 1 step) = 0.4

$p\{X_2=1\mid X_1=2\}$ = $p\{X_3=1\mid X_2=2\}$ = $p\{X_4=1\mid X_3=2\}$ = P(weather goes from $Sunny$ to $Rainy$ in 1 step) = 0.2

$p\{X_2=2\mid X_1=2\}$ = $p\{X_3=2\mid X_2=2\}$ = $p\{X_4=2\mid X_3=2\}$ = P(weather goes from $Sunny$ to $Sunny$ in 1 step) = 0.8

The above is defined by the matrix you provided. Hope this helps!

ydubey7
  • 146
0

It depends on if we assume row vectors or column vectors. If using a column vector $(p_s , p_r)^T$ of todays probabilities, then 0.6 is the probability that tomorrow is sunny given that today is sunny, 0.4 is the probability that tomorrow is rainy given that today is sunny. 0.2 that tomorrow is sunny if today is rainy and 0.8 that tomorrow is rainy if today is rainy.

If we assume row vectors $(p_s,p_r)$ to multiply from the left, however.. the matrix will be transposed to have the same meaning because of the matrix algebra for transposition of matrix multiplication: $({\bf Ap})^t = {\bf p}^t{\bf A}^t$

mathreadler
  • 25,824