1

If $X$ is a discrete Markov chain with state space $S=\{1,2\}$ and transition matrix

\begin{equation*} P=\begin{pmatrix} 1-a& a\\ b& 1-b \end{pmatrix}. \end{equation*}

I must answer the question "Classify the states of the chain". What is meant by this? Must I say if the states are recurrent or transient? And if so, which one is it?

  • Without seeing the context, I guess I would assume they want to know recurrent or transient. I'm assuming you know the definition of those terms; what do you think the states are? – Brian Tung May 05 '15 at 20:29
  • There is no further context given, and I think recurrent, but I'm not sure –  May 05 '15 at 20:31
  • Yes, under fairly broad assumptions ($0 < a, b \leq 1$), the states are recurrent. If both $a = b = 1$, the states are periodic. If only one of them equals $1$, then the states are merely recurrent. – Brian Tung May 05 '15 at 20:36

1 Answers1

1

If $0<a,b\leqslant1$ then $P_{ij}+P^2_{ij}>0$ for all $i,j$ so the Markov chain is (positive) recurrent. You can verify this by computing $\mathbb E_i[\tau_i]$ where $$\tau_i = \inf\{n>0 : X_n=0\},$$ and $\mathbb E_i[\cdot]$ denotes conditioning on $X_0=i$.

If $a=0$ (resp. $b=0$) then state $1$ (resp. state $2$) is absorbing, and therefore transient.

Math1000
  • 36,983