1

We have a biased coin which, if tossed, shows heads (H) with probability 1/3 and tails (T) with probability 2/3. First, we will toss the coin 3 times. We will model this triple of tosses as well as further tossing of the coin by an MC with states 1,2,...,8. The interpretation of state 1 is the last three observed outcomes were TTT, the interpretation of state 2 is the last three observed outcomes were TTH, ..., the interpretation of state 8 is the last three observed outcomes were HHH. The MC should correspond to the transitions between individual states in one toss. Construct the graph and the transition matrix Π of the MC. Find the matrices Π(2) and Π(3) of transition probabilities after 2 and 3 steps. Find the stationary distribution of the MC.

Hi, I am stuck in this question. I am unable to make its transition matrix. Please give me some hints. Thanks in Advance

Saria Ahmad
  • 11

1 Answers1

1

Each column of the transition matrix corresponds to one of the pure states, and describes the probabilities of what comes next if we start in one of those pure states. For example, the first state $TTT$ can lead to $TTT$ if we flip tails (probability $\frac23$) or $TTH$ if we flip heads (probability $\frac13$), and the other six states are impossible. The first column of the matrix will thus have one entry equal to $\frac23$, another equal to $\frac13$, and the other six equal to zero (Which ones? That's what the numbers on the states are for). Repeat for the other seven columns, and that'll be the transition matrix.

Some common-sense details:

  • Every entry in the transition matrix must be nonnegative, and the sum of the entries in a column must be $1$. We go somewhere with probability $1$, after all.

  • In this coin-flipping process, every pure state leads to two possibilities. Each column should have exactly two nonzero entries, corresponding to the possible results of the next flip.

jmerry
  • 19,403