5

I was watching Harvard STAT 110 lecture series on Youtube. As per lecture 10, given $$T=X+Y$$ I have the following question:

Why is $$P(T=t) = \sum P(T=t|X=x)\ P(X=x)$$ and not $$\sum P(T=t|X=x)\ P(X=x)+ \sum P(T=t|Y=y)\ P(Y=y)?$$

Karn Watcharasupat
  • 4,323
Anisha Jain
  • 71
  • 5
  • 1
    @layman The relation $T=X+Y$ is not relevant in this matter. What follows are "why is..." is also true without that relation. The only things that matter are that $T$ and $X$ must be defined on the same probability space and that $X$ must be discrete. – drhab May 04 '18 at 14:24
  • 1
    @layman Last comment: "...so running through all possible values of $T$ and $X$ is enough..." Enough for what?? It makes no sense at all. – drhab May 04 '18 at 14:35
  • 1
    @layman I don't know what a "stretch" is. Thank you for telling me that you did not downvote me. My comment on your comment concerns the fact that the relation $T=X+Y$ (and all that goes together with it) is completely irrelevant in this situation. $P(T=t)=\sum_xP(T=t\mid X=x)P(X=x)$ is always true under the conditions that I mentioned. Let me say (more careful) that your comment makes no sense to me then. – drhab May 04 '18 at 16:04
  • 1
    @layman While what you've said is true (the value of $Y$ is determined by the values of $T$ and $X$), it is irrelevant to the question. So presenting it as an explanation here is extremely misleading. – Alex Kruckman May 04 '18 at 19:36
  • 1
    @layman I'm sorry, but you're wrong. drhab has explained the situation perfectly. – Alex Kruckman May 04 '18 at 20:06
  • @AlexKruckman maybe I'm giving a charitable interpretation, but layman's comment seems to say exactly what quasi's answer below says. – Tyberius May 04 '18 at 20:16
  • @Tyberius Maybe prior to quasi's edit in response to my comment? – Alex Kruckman May 04 '18 at 20:25
  • Is it worth mentioning that X + Y is a sum of the values, not the probabilities? There are two completely different types of numbers in these expressions, which can be confusing to a beginner. – Charles Gillingham May 04 '18 at 20:26

4 Answers4

8

Presumably you meant $T=X+Y$, not $Y=X+Y$.

The cases for $X$ are exhaustive, since $X$ must take some value.

Analogously, the cases for $Y$ are exhaustive.

Thus, if you case it by values of $X$, you get $$P(T=t) = \sum P(T=t|X=x)\,P(X=x)$$ and if you case it by values of $Y$, you get $$P(T=t) = \sum P(T=t|Y=y)\,P(Y=y)$$ but if you sum both, you would get twice the correct answer.

quasi
  • 58,772
  • 1
    I like this answer, except for the word "since". The cases for $X$ are exhaustive because we're summing over all possible values of $X$! The fact that $T = X+Y$ is completely irrelevant. – Alex Kruckman May 04 '18 at 19:39
  • 1
    @Alex Kruckman: I agree. I'll edit to reflect that idea. – quasi May 04 '18 at 20:02
  • +1 Now it is the time for upvoting. Nice answer! – drhab May 04 '18 at 20:45
  • 2
    @layman: Some value of $X$ must happen, so that already exhausts all cases, regardless of whether or not the value of $Y$ is forced. Similarly, the cases could be based on $Y$, or even something else, as long as the cases are exhaustive and mutually exclusive. So either of the two sums gives the total probability for a given value of $T$, hence, summing both would yield twice the correct result. – quasi May 04 '18 at 21:13
  • 1
    No, the sum $$\sum_{x,y} P(T=t|X=x, Y=y),P(X=x, Y=y)$$ is not double summing. Double summing is what the OP tried to propose, namely $$\sum P(T=t|X=x)\ P(X=x)+ \sum P(T=t|Y=y)\ P(Y=y)$$ and my answer explains why that would be twice the correct result. – quasi May 04 '18 at 21:46
  • 1
    @layman: It really isn't double summing. As an example, let the experiment be the rolling two standard dice. Let $X$ be the value of the first die, let $Y$ be the value of the second die, and let $T=X+Y$. Then we have $$P(T=5) = \sum P(T=5|X=x),P(X=x)$$ and $$P(T=5) = \sum P(T=5|Y=y),P(Y=y)$$ but we also have $$P(T=5)=\sum_{x,y} P(T=5|X=x, Y=y),P(X=x, Y=y)$$ – quasi May 04 '18 at 22:04
  • 1
    @layman: For the dice example, test the claim $$P(T=5)=\sum_{x,y} P(T=5|X=x, Y=y),P(X=x, Y=y)$$ Note that for each of the $36$ summands in the above sum, we always have $$P(X=x, Y=y) =\frac{1}{36}$$ and the factor $$P(T=5|X=x, Y=y)$$ is either $1$ or $0$, according as $x+y=5$ or $x+y\ne 5$. – quasi May 04 '18 at 22:14
  • @layman "that means that you have the same terms being added twice..." No! As quasi made perfectly clear the only effect is that there are more terms and that lots of them take value $0$. The outcome is not affected. I hope that you are convinced now. I would not say that you are going crazy, but just that you bothered by a blind spot. It happens to me on regular basis. – drhab May 05 '18 at 06:05
  • @drhab you’re right, it was a blind spot. – layman May 05 '18 at 13:48
4

Note that $T=X+Y$ so when $T=t,X=x$ you have $Y=t-x$, in other words, \begin{align} P(T=t)& = \sum_{x} \left[{P(T=t|X=x)\cdot P(X=x)}\right]\\ &=\sum_{x} \left[{P(X+Y=t|X=x)\cdot P(X=x)}\right]\\ &=\sum_{x} \left[{P(Y=t-x|X=x)\cdot P(X=x)}\right] \end{align} which means you have already unknowingly taken $Y$ into account!

Karn Watcharasupat
  • 4,323
1

Observe that for discrete random variables $X$ and $Y$: $$\{T=t\}=\bigcup_x\{T=t\wedge X=x\}=\bigcup_y\{T=t\wedge Y=y\}$$ where $x$ ranges over all values that are taken by $X$ and $y$ over all values that are taken by $Y$.

Then: $$\sum_x P(T=t\mid X=x)P(X=x)+\sum_y P(T=t\mid Y=y)P(Y=y)=$$$$\sum_x P(T=t\wedge X=x)+\sum_y P(T=t\wedge Y=y)=P(T=t)+P(T=t)=2P(T=t)$$

Alex Kruckman
  • 76,357
drhab
  • 151,093
  • Explain the downvote please. – drhab May 04 '18 at 14:17
  • I didn't vote but I supposed it's partly because the notation you used too. I don't even know how to interpret some and the OP seems to be at a very early stage of learning probability & stats... – Karn Watcharasupat May 04 '18 at 14:56
  • @KarnWatcharasupat Thank you for your reaction. Could you tell me what in my notation is unclear for you (so that I can take that into account in my later answers to questions)? – drhab May 04 '18 at 15:23
  • It's not unclear actually. More like it's something students early in stats course wouldn't have seen or be able to read, but after looking them up then it's fine e.g. the big caps and the wedges. (I only just finished A-level a few months ago) – Karn Watcharasupat May 04 '18 at 15:25
  • @KarnWatcharasupat Thank you and the best wishes for your study of probability. – drhab May 04 '18 at 15:29
  • @AlexKruckman Thank you for editing and repairing my answer. – drhab May 05 '18 at 06:10
1

@quasi's answer is an excellent rigorous answer, but if you'd like a more intuitive way to look at it:

Suppose you want to get to a street corner that is north-east of your current location. You can either go one block north and one block east or go one block east and then one block north, but if you do both of them you'll overshoot your target. I.e. you can use either of the two sums and be valid, but you can't do both (i.e. add them together).

JonathanZ
  • 10,615