0

I have a very simple Markov Chain and would like to know if the following question is possible to answer?. I have scoured the internet and could not find much application of Markov Chains in answering this question. Any help or leads in the right direction will be greatly appreciated. So here goes....

Question: Given that State A transitions to State B 10% of the time and stays in State A 90% of the time, and also given that for each transition of A -> A, a 20 second time penalty will be accrued before another transition occurs..."What is the probability that the system will transition from State A to State B by time 50 seconds given that the clock starts at 0 seconds"

Thanks for the help in advance

Simplified Markov Chain with 1 Absorbing State

Justin Yapp
  • 1
  • So you just want the probability that a transition to state B occurs at $0$ seconds, $20$ seconds, or $40$ seconds, isn't that right? Calculate each probability and add them up. – saulspatz Oct 03 '20 at 22:39
  • Consider 50 seconds to be a Hard Deadline constraint, I want to calculate the probability of missing the 50 second deadline. – Justin Yapp Oct 03 '20 at 22:44
  • That's one minus the probability of making it, right? – saulspatz Oct 04 '20 at 01:15

1 Answers1

0

If you fail to transition to state B the first three chances you get, $60$ seconds will pass and you will miss the deadline. The probability of this is $$.9^3=.729$$

saulspatz
  • 53,131