1

I'm working my way through Norris's classic textbook, but I'm having problems with this hitting probability question:

"A gambler has £2 and needs to increase it to £10 in a hurry. He can play a game with the following rules: a fair coin is tossed: if a player bets on the right side, he wins a sum equal to his stake, and his stake is returned; otherwise he loses his stake.

"The gambler decides to use a bold strategy in which he stakes all his money if he has £5 or less, and otherwise stakes just enough to increase his capital, if he wins, to £10. Let X_0=2 and let X_n be his capital after n throws.

"Prove that the gambler will achieve his aim with probability 1/5."

How does one solve this?

Ubt388
  • 37
  • 1
  • 3

2 Answers2

4

Here is a martingale (not a markov chain) solution that comes from noticing that he's playing a fair game, i.e., if $X_n$ is his money at time $n$ then $E(X_{n+1}|X_n)=X_n$.

By the optional stopping theorem, the expected value when the game is finished is equal to the expected value when the game starts. He ends with either $\$0$ or $\$10$ (sorry, but I don't know how to make the pound symbol), and if $p$ is the probability that he ends in $\$10$, then $0(1-p)+10p=E(0)=2$, or $10p=2$.

Aaron
  • 24,207
2

This looks to me as an absorbing Markov process. He starts at 2 GBP (forgive me for the denomination), if he wins he goes to 4, then to 8 and possibly to 10. But from 8 he can go back to 6 and from 6 back to 2 or wins to 10. From 2 he can also lose to 0. So we have the following "states": 0, 2, 4, 6, 8, and 10 where 0 and 10 are the absorbing states. From each state you can write down the probability according to the chances by the coin. Your matrix is a 6by6 matrix. If we partition this matrix in 4 sub matrices, we have a 2by2 matrix top left which is the Identity matrix (that's where the states "zero" and "10" are→game over). It's the top right matrix and bottom right matrix we need. According to the theories of Markov, the answer lies in the result of $R(I-Q)^-1$ where R is the top right matrix and Q the bottom right. The probability from the state "2" to "10" should give you the 1/5

imranfat
  • 10,029
  • I worked it out. $R(I-Q)^-1$ has a 0.2 in the first column, second row, indicating the probability from state 2 to state 10 and so your assertion about 1/5 is indeed correct! This is a great problem, thanks for posting as I am going to "borrow" it from you :) – imranfat Nov 19 '13 at 20:36
  • $\it\unicode{xA3}$ – Did Nov 23 '13 at 09:31
  • Could you actually show how to compute the probability $1 \over 5$? –  Jan 23 '20 at 17:07
  • @user6969 It's not computed "manually". It can be read from $R(I-Q)^{-1}$ where you look in the column from $2$ to $10$. You can cheat by putting the actual matrix in the TI and raise it to a very high power. The answer is incredibly close to 0.2 – imranfat Jan 23 '20 at 18:14