MY ATTEMPT AT SOLUTION: $P(win)=P(draw.at.least.3. red)=P(draw 3 R)+P(draw 4 R) + P(draw 5 R)$
Using the law of total probability:
$=P(bin1)*P(draw.3|bin1)+P(bin2)*P(draw.3|bin2)+P(bin3)*P(draw.3|bin3)+P(bin4)*P(draw.3|bin4)+P(bin5)*P(draw.3|bin5)+P(bin1)*P(draw.4|bin1)+P(bin2)*P(draw.4|bin2)+P(bin3)*P(draw.4|bin3)+P(bin4)*P(draw.4|bin4)+P(bin5)*P(draw.4|bin5)+P(bin1)*P(draw.5|bin1)+P(bin2)*P(draw.5|bin2)+P(bin3)*P(draw.5|bin3)+P(bin4)*P(draw.5|bin4)+P(bin5)*P(draw.5|bin5)$
Factoring this all out I simplify to get:
$P(bin1)*(P(draw.3|bin1)+P(draw.4|bin1)+P(draw.5|bin1)) +P(bin2)*(P(draw.3|bin2)+P(draw.4|bin2)+P(draw.5|bin2)) +P(bin3)*(P(draw.3|bin3)+P(draw.4|bin3)+P(draw.5|bin3) +P(bin4)*(P(draw.3|bin4)+P(draw.4|bin4)+P(draw.5|bin4)) +P(bin5)*(P(draw.3|bin5)+P(draw.4|bin5)+P(draw.5|bin5))$
Since we are given the ratios of the sizes of the bins I tried to find the probability that way. So bin 5 is the smallest.
$P(bin1)=5*P(bin5)$
$P(bin2)=4*P(bin5)$
$P(bin3)=3*P(bin5)$
$P(bin4)=2*P(bin5)$
$P(bin5)=1*P(bin5)$
since the probabilities of all the bins need to add to 1 we have
$1=15*P(bin5) \implies P(bin5)=\frac{1}{15}$
Then I can calculate the remainder probabilities of bins.
$P(bin1)=1/3$
$P(bin2)=4/15$
$P(bin3)=1/5$
$P(bin4)=2/15$
$P(bin5)=1/15$
Then calculating $P(draw.3|bin1)=\frac{{12\choose 3}{13\choose 2}}{{25 \choose 5}}$
So I did these calculations for the remainder probabilities and got something around 0.58.
So the probability of winning 10 is 0.58 and the probability of losing $p$ is 0.42.
So then I set $E[X]=0$ and solved for how much you would lose if playing the game.
$10(0.58)+p(0.42)=0$
$p=-13.8$
This makes no sense. I'm losing more than I would win by playing the game?
I have a feeling I did this question all wrong. Can someone please help me with the solution?