Probability of a statement being true vs honesty of people involved.

Question

Problem statement 1: $A$ and $B$ are two independent witnesses (that is, there is no collusion between them) in a case. The probability that $A$ will speak the truth is $x$ and the probability that $B$ will speak the truth is $y$. $A$ and $B$ agree in a statement. What is the probability that the statement is true?

I want to solve this problem using Bayes' theorem.
Clearly, since $A$ and $B$ are witnesses, if they speak the truth then the statement is truth.
Let $X_t$ be the event that person $X$ tells the truth. Let $P(A\cap B) $ be the probability that $A$ and $B$ agree on a statement (both may either agree by telling the truth or lie) Let $S$ be the event that the statement is true. Let $P(S)=p$.
We need to find probability that the statement is true under the condition that $A$ and $B$ agree on the statement, i.e., $P(S_t|A\cap B)$. Now by Bayes' theorem, we have:

$P(S_t|A\cap B)=\frac{P(S_t)P(A\cap B|S_t)}{P(A\cap B)}=\frac{pxy}{P(A_t\cap B_t)+P(A_f\cap B_f)}=\frac{pxy}{xy+(1-x)(1-y)}? \tag{1}$

Problem Statement 2:Same as above problem except that $A$ and $B$ are not actual witnesses rather they were near (say around 20 km) to the location where accident happened. In this case, their honesty has nothing to do with whether the statement is true or not. Same as above, in this case also, we get

$P(S_t|A\cap B)=\frac{P(S_t)P(A\cap B|S_t)}{P(A\cap B)}=\frac{pxy}{P(A_t\cap B_t)+P(A_f\cap B_f)}=\frac{pxy}{xy+(1-x)(1-y)}? \tag{2}$

My doubts:
P. I took a reference of this answer What is the probability of two people telling the truth? and this answer How to find the probability of truth?. It seems that $(1)$ should be
$$P(S_t|A\cap B)=\frac{pxy}{P(A_t\cap B_t)+P(A_f\cap B_f)}=\frac{pxy}{pxy+(1-x)(1-y) (1-p)}.$$ I don't understand why?

Q. If the expression in (P) above, is correct, would it be correct to say that $p=1/2$ as whatever the statement be, it has to be either true or false! And it is not surprising as we are, in $(1)$ finding the validity of the statement under the condition that $A$ and $B$ agree on the statement. Does this mean that answer here How to find the probability of truth? is wrong as it says "$xyp+(1-x)(1-y)(1-p)=(1-x-y+2xy)p$". But this is not true unless $p=0.5$?
R. What about problem $(2)$? Is the expression for the same in $(2)$ correct?

Please help me understand. Thanks.

Answer 1

The thing about finding conditional probabilities is that the answer can change depending on what you're conditioning on, and one can frame the same empirical facts as different observations, resulting in different answers. A famous example is the Monty Hall problem, in which everyone agrees that what you see is a goat, but the answer is different depending on how one frames the event that one is conditioning on. For instance, suppose Door A is chosen, and Door B is opened, revealing a goat. If one frames the question "What is the probability of A having a car, conditioned on B having a goat", the answer is 1/2. If it's framed as "What's the probability of A having a car, conditioned on at least one of the doors other than A having a goat?", the answer is 1/3.

Getting back to your problem, suppose $A$ and $B$ both claim that a traffic light was red. If we frame the question as "What's the probability that the traffic light was actually red, conditioned on the events '$A$ says it was red' and '$B$ says it was red'?", then the answer will be $\frac{pxy+(1-p)(xy)}{pxy+(1-p)(xy)+p(1-x)(1-y)+(1-p)(1-x)(1-y)}$. However, if you're simply asking for the probability that they would agree on a true answer, without including what that answer was in the condition, then you don't need to know what $p$ is; the probability of them agreeing on a true answer is $\frac {xy}{xy+(1-x)(1-y)}$.

If you want a rigorous proof of that, consider what the success space is, and what the population space is. The success space consists of two spaces: the light was red and they both say it's red, or it wasn't red, and they both say it wasn't. The population space has four events: red and they both say red, red and they both say not red, not red and they both say red, and not red and they both say not red. The probability is the sum of all the success probabilities divided by the sum of all the population probabilities. This gives

$\frac {pxy+(1-p)xy}{pxy+p(1-x)(1-y)+(1-p)(xy)+(1-p)(1-x)(1-y)}$

The numerator can be simplified to just $xy$. The denominator can be rewritten as $$p(xy+(1-x)(1-y))+(1-p)((xy)+(1-x)(1-y)) =$$ $$(p+(1-p))(xy+(1-x)(1-y)) =$$ $$xy+(1-x)(1-y)$$. Thus, the final probability is $\frac {xy}{xy+(1-x)(1-y)}$.

As for your second question, it's not clear what you're asking.

Answer 2

Frankly, I don't see how any of this is correct. First off I agree with Lulu, the second problem statement makes no sense without additional details. It's (somewhat) common knowledge that eyewitness testimony is not very reliable. Is this because the eyewitnesses are lying? No, it's because they misremembered or interpreted something incorrectly. So we have to also know what the probability is that they even know what they saw is "true" or "false".

Correction

As for Problem Statement 1: What is the probability of two people telling the truth? is asking a different question.

It's forcing a weird situation that is unintuitive. It's saying an event happened or didn't. So what's the probability that the event happened if the two people agree (even if they agreed it didn't happen!)? This framing is very different from your question, which is if two people agree, are they telling the truth?

end of correction

There is no $p$, the probability of the thing actually being true or not. Let me give you an example, let's say I have a weighted coin that comes up heads 75% of the time and tails 25% of the time. If I flip the coin and it comes up heads, A will say it's heads with $p = x$ and B will say it's heads with $p = y$. The probability of them telling the truth has nothing to do with the probability of the outcome.

Now if I change and say what is the probability that $A$ and $B$ will agree the coin is heads, then yes, the $p$ comes into play (and it's more complicated because now there's a 75% chance it is heads and they both tell the truth or it's tails and they both lie).

So, this is a pretty simple problem, there are four possibilities:

1. Both tell the truth: $p = xy$
2. Both lie: $p = (1 - x)(1 - y)$
3. A lies, B tells truth: $p = (1 - x)y$
4. B lies, A tells truth: $p = x(1 - y)$

There are only two cases (1. and 2.) where A and B agree, so this is the "universe" of possibilities. Therefore, if they agree, the probability of them telling the truth is the probability of them both telling the truth divided by the probability that they both agree:

$$ P\left(\text{Truth} | \text{agree}\right) = \frac{xy}{xy + (1- x)(1 - y)} $$

The above is using the direct definition of a conditional probability (which in my opinion is more appropriate for this problem). It can be tricky to relate this to Baye's Theorem:

$$ P\left(\text{Truth} | \text{agree}\right) = \frac{P\left(\text{agree}|\text{Truth}\right)P\left(\text{Truth}\right)}{P\left(\text{agree}\right)} $$

The problem is that $P\left(\text{Truth}\right)$ is an abbreviation. It's not the probability that the thing actually happened, it's the probability that they're both telling the truth.

So we know the probability that they both tell the truth: $p = xy$. But what's the probability that they agree if they both tell the truth? Think about that in plain English. If they both tell the truth, they agree, don't they? So $P\left(\text{agree}|\text{Truth}\right) = 1$.

Addressing Comments (which Accumulation already addressed)

Let's look at one specific possible outcome: a red ball is drawn--what is A's response?

He will say, with probability $x$ that a red ball was drawn. What is the probability he'll say not red? Obviously $1 - x$. Now right away, does it matter what the other colors are? What if the other colors are green, blue, yellow, and purple? What is the probability he'll say the drawn ball was brown? There are (more or less) infinite numbers of colors he could say as a lie--a lie isn't bound by what is possible.

But OK, let's say the balls are red, green, blue, yellow, and purple and he always says one of those. What are we told? We're told that he tells the truth with probability $x$. Furthermore, let's say that A's favorite colors are, in this order: purple, yellow, blue, green, and red. So, if A lies, he'll choose his favorite color according to that ranking (probabilistically of course). Let's go through the possibilities:

1. Red Ball is chosen

He either tells the truth ($p = x$) or he says it's purple with $P(purple | lie) = 0.4$, yellow with $P(yellow | lie) = 0.3$, blue with $P(blue|lie) = 0.2$, and green with $P(green|lie) = 0.1$.

2. Green Ball is chosen

He either tells the truth ($p = x$) or he says it's purple with $P(purple | lie) = 0.4$, yellow with $P(yellow | lie) = 0.3$, blue with $P(blue|lie) = 0.2$, and red with $P(red|lie) = 0.1$.

3. Blue Ball is chosen

He either tells the truth ($p = x$) or he says it's purple with $P(purple | lie) = 0.4$, yellow with $P(yellow | lie) = 0.3$, green with $P(green|lie) = 0.2$, and red with $P(red |lie) = 0.1$.

4. Yellow Ball is chosen

He either tells the truth ($p = x$) or he says it's purple with $P(purple | lie) = 0.4$, blue with $P(blue| lie) = 0.3$, green with $P(green |lie) = 0.2$, and red with $P(red|lie) = 0.1$.

5. Purple Ball is chosen

He either tells the truth ($p = x$) or he says it's yellow with $P(yellow | lie) = 0.4$, blue with $P(blue | lie) = 0.3$, green with $P(green |lie) = 0.2$, and red with $P(red|lie) = 0.1$.

You might assert that if he says the ball is red, that there's a $p = x$ chance of him telling the truth (as I initially, erroneously, did) but that's actually wrong (and I show why below).

1. Ball is Red: A tells truth: $p = x$
2. Ball is Green: A lies, $p = 0.1(1 - x)$
3. Ball is Blue: A lies, $p =0.1(1 - x)$
4. Ball is Yellow: A lies, $p = 0.1(1 - x)$
5. Ball is Purple: A lies, $p = 0.1(1 - x)$

So now the probability of each of these scenarios (red ball, green ball, etc.) is $\frac{1}{5}$ so we have:

\begin{align*} P(\text{ball is red} | \text{A says it's red}) =&\ \frac{\frac{1}{5}x}{\frac{1}{5}\left(x + 4\cdot 0.1(1 - x)\right)} \\ =&\ \frac{x}{x + 0.4 - 0.4x} \\ =&\ \frac{x}{0.6x + 0.4} \end{align*}

But this is not the same as "is A telling the truth?". This is only answering the probability that "if A says red, is he telling the truth?" This is very analogous to my initial example where I say there's a difference between "what is the probability that they say heads vs. tails" vs. "what is the probability of lying".

Above I showed that $P(\text{A says R}) = \frac{1}{5}\left(0.6x + 0.4\right)$, likewise we can find the rest:

\begin{align*} P(\text{A says G}) = \frac{1}{5}\left(x + (0.1 + 3\cdot0.2)(1 - x)\right) =&\ \frac{1}{5}\left(0.3x+ 0.7\right) \\ P(\text{A says B}) = \frac{1}{5}\left(x + (2\cdot0.2 + 2\cdot0.3)(1 - x)\right) =&\ \frac{1}{5} \\ P(\text{A says Y}) = \frac{1}{5}\left(x + (3\cdot0.3 + 0.4)(1 - x)\right) =&\ \frac{1}{5}\left(1.3 - 0.3x\right) \\ P(\text{A says P}) = \frac{1}{5}\left(x + 4\cdot0.4(1 - x)\right) =&\ \frac{1}{5}\left(1.6 - 0.6x\right) \\ \end{align*}

When you add all of these up, you find that the probability is $1$ (i.e. the probability that they pick a color is $1$) as it should be.

Answer 3

I'm writing this answer because there seems to be a lot confusion involved. The other MSE questions in the post don't all solve the same problem.

Let us set the problem up. $A$ and $B$ are two people who happen to witness an event, say person X perform a backflip. The probability that $A$ says the truth is to be interpreted as follows: If $A$ witnesses $X$ perform a successful backflip 100 times, he would only say that $X$ performed it $100x$ times. Even if $X$ fails the backflip $100$ times, $A$ would still $X$ only failed it only $100x$ times and actually landed the rest. $A$ always has a chance to lie regardless of whether $X$ landed the backflip or not.

Here, we want to calculate the probability that $X$ landed his backflip provided we know that $A$ and $B$ say the same thing about the event. Either $A$ and $B$ say that $X$ landed it or he doesn't. So, what is the probability that $X$ lands the flip when $A,B$ said the same thing? For this, one can refer to this question.

In What is the probability of two people telling the truth? , the probability they try to find is different. Imagine you know that $X$ only lands the backflip with a probability of $p$. You couldn't go see it for yourself, so you ask your friends $A$ and $B$. Now, what is the probability that $X$ landed his backflip given that $A$ and $B$ say that he landed the backflip? The link answers exactly that.

Note how the two problems are different. The first one calculates the probability that $X$ landed the flip when $A,B$ agree on their answer, that he did land it or that he didn't. The second one assumes that $A,B$ both say that $X$ indeed landed his flip.

In your calculation labelled $(1)$, you have forgotten a few cases. There should be 4 terms in the denominator.

If $A$ and $B$ agree:

1. they can agree that $X$ landed his backflip when he landed his backflip.
2. they can agree when $X$ didn't land it, when he actually landed the flip.
3. they can agree that $X$ didn't land it, when he didn't land it.
4. they can agree that $X$ landed it, when he didn't land it.

If you make this correction, then your calculation becomes correct and will coincide with How to find the probability of truth?.

In your problem statement 2, you have chosen that $A$ and $B$ don't actually see the event. Then, any statement by $A$ or $B$ is actually not a trustworthy statement even if it agrees with the truth. So, we are not able to use the probability of their 'faithfulness' here.