2

I have a question I am working on:

Suppose $\{Z_n, n ≥ 1\}$ are iid outcomes of successive throws of a fair die. Then, let $X_n = \max\{Z_1, ..., Z_n\}$. It is easy to show that $X_n$ is Markov.

I am trying to find the higher powers of $P$, which are the $n$-step transition probability matrices of $P$.

So far, I have broken the problem down into three cases:

The probability I would like to find is $p^{n}(i,j)$, defined as being the probability of going from $i$ to $j$ in $n$ steps.

So far I have found the case for $i=j$ and $i>j$.

Where I am stuck is $i<j$. I know that the correct answer should be:

Suppose $i < j$. Then $p^{n}(i,j) = \frac{(j^{n}-(j-1)^{n})/6^{n+1}}{1/6} = \frac{j^n - (j-1)^n}{6^n}$.

However, I am not sure why the answer is supposed to be so. Would anyone be kind enough to help explain it to me? Thank you!

Did
  • 279,727
user123276
  • 3,445

1 Answers1

2

Let $i\lt j$. Then $p^n(i,j)$ is the probability that $X_{k+n}=j$ conditionally on $X_k=i$, for every $k$, that is, the probability that $\max\{X_k,Z_{k+1},\ldots,Z_{k+n}\}=j$ conditionally on $X_k=i$, that is, the probability that $Y_n=j$ conditionally on $X_k=i$, where $Y_n=\max\{Z_{k+1},\ldots,Z_{k+n}\}$.

Equivalently, $p^n(i,j)=P(Y_n=j)$ since $Y_n$ is independent of $X_k$. Additionally, for every $j$ in $\{1,2,\ldots,6\}$, $P(Y_n\leqslant j)=P(Z\leqslant j)^n=\left(\frac{j}6\right)^n$. Can you finish this?

Did
  • 279,727