Is this an absorbing Markov chain?

Question

I started to study Markov chains and one of the exercices that I found on the internet is the following one:

A forest consists of two types of trees: those that are 0-5 ft and those that are taller than 5 ft.

Every year, 40% of all 0-5 ft tall trees die, 10 % are sold for 20 dollars each, 30% stay between 0-5 ft, and 20% grow to be more than 5 ft Each year, 40% of all trees taller than 5 ft are sold for 50 dollars, 20% are sold for 30 dollars, 10% die and 30% remain in the forest.

a. What is the probability that a 0-5-ft tall tree will die before being sold?

b. If the cost for plant a tree (less than 5 ft) is 5 dollars what is the expected revenue earned from that tree?

I was thinking to consider this as a markov chain with state space = {0-5 ft, >5 ft, die, sold 30\$, sold 50\$, sold 20\$}. die, sold 30\$, sold 50\$, sold 20\$ would be absorbing states.

Is my approach correct?

Any hint how to solve a?

Answer 1

Yes, your approach can be used to answer both questions.

For the first question you need to find the probability $\ d_1\ $ that a tree will end up in state $3$ (i.e. dead) given that it is currently in state $1$ (i.e. from zero to five feet high). But since a tree of $0$-$5$ ft can die either before or after growing to more than $5$ ft, you also have to find the probability $\ d_2\ $ that a tree will end up in state $3$ given that it is currently in state $2$.

The probability that a tree in state $2$ eventually ends up in state $3$ is the sum of the probabilities that it dies this year (namely, $\ 0.1\ $) and the probability that it remains in the forest until next year (that is, remains in state $2$—which is $\ 0.3\ $) but subsequently ends up in state $3$, which has probability $\ d_2\ $. Thus $\ d_2\ $ must satisfy the equation $$ d_2=0.1+0.3d_2\ . $$ Can you see how to derive a similar equation for $\ d_1\ $ ? Once you have the right equation, you shouldn't have any trouble solving the two equations for $\ d_1\ $ and $\ d_2\ $.