What is the probability of two people telling the truth?

Question

I have a similar problem described in this question: How to find the probability of truth?

The question reads:

A and B are independent witness in a case. The probablity that A speaks the truth is 'x' and that of B is 'y'.If A and B agree on a certain statement, how to find the probability that the statement is true ?

In my problem, I also know that the probability for the statement to be true is $p$.

The accepted answer starts with:

Let:

$A_t$ stand for "A says statement is true." and $A_f$ for "A says statement is false" and

$B_t$ stand for "B says statement is true." and $B_f$ for "B says statement is false" and

$S_t$ stand for "Statement is true" and $S_f$ for "Statement is false" and

Then, we know that:

$\text{Prob}(A_t | S_t) = \text{Prob}(A_f | S_f) = x$, and

$\text{Prob}(A_t | S_f) = \text{Prob}(A_f | S_t) = 1-x$, and

$\text{Prob}(B_t | S_t) = \text{Prob}(B_f | S_f) = y$, and

$\text{Prob}(B_t | S_f) = \text{Prob}(B_f | S_t) = 1-y$, and

My question is: aren't the equations above wrong?

Shouldn't $\text{Prob}(A_t \cap S_t) = \text{Prob}(A_f \cap S_f) = x$?

Shouldn't $\text{Prob}(A_t | S_t) = \frac{\text{Prob}(A_t \cap S_t)}{\text{Prob}(S_t)} = \frac{x}{p}$?

As I understand it, "A speaks the truth" means that A agrees on the statement and the statement is true, i.e. "A speaks the truth" is the intersection between $A_t$ and $S_t$.

Or, equivalently, "A speaks the truth" means that A doesn't agree and the statement is false, i.e. it is the intersection between $A_f$ and $S_f$.

UPDATE: with a numerical example, I found out that I'm wrong and the answer may be right. I found out that for certain $x$ and $p$ (from 0 to 1), $x / p$ may be greater than 1, therefore it can't be a probability. So, where is the fallacy in my thought?

Answer 1

I will construe the terms of the problem as follows: Someone (let's say, the judge) makes a statement. Both $A$ and $B$ are independently asked if they agree with that statement (in other words, if it is true).

There are two possibilities: either the statement is true (with probability $p$), or the statement is false (with probability $1-p$).

If the statement is true, both $A$ and $B$ will agree with that statement with probability $xy$. If the statement is false, both $A$ and $B$ will agree with it with probability $(1-x)(1-y)$.

Thus the probability that a statement is true, given that both $A$ and $B$ agree with it, is

$$ P(\mbox{true} \mid A, B \mbox{ agree with it}) = \frac{xyp}{xyp+(1-x)(1-y)(1-p)} $$

ETA: Regarding the fallacy in your logic: The probability $x$ of $A$ being honest does not denote the joint probability that a statement is true and that $A$ agrees with it; it is the conditional probability that $A$ agrees with a statement, given that it is true. Symbolically:

$$ x = P(A_t \mid S_t) = \frac{P(A_t, S_t)}{P(S_t)} \not= P(A_t, S_t) $$

Answer 2

The confusion is in wording here. "A statement" isn't a pure boolean. The event definitely did or didn't happen with probability $p$ and $1 - p$, respectively. But "statement" is related to the event:

\begin{align*} \text{if $p$ then statement says $p$ with $prob = x$} \\ \text{if $\neg p$ then statement says $\neg p$ with $prob = x$} \end{align*}

"Statement" means "you told the truth". So if "the thing" didn't happen, you'll say it didn't--truthfully. This gives the following truth table:

\begin{align*} \text{Event happens, both tell truth}&\ p_{TTT} = pxy \\ \text{Event happens, second lies}&\ p_{TTF} = px(1 - y) \\ \text{Event happens, first lies}&\ p_{TFT} = p(1 - x)y \\ \text{Event happens, both lie}&\ p_{TFF} = p(1 - x)(1 - y)\\ \text{Event doesn't happen, both lie}&\ p_{FTT} = (1 - p)(1 - x)(1 - y) \\ \text{Event doesn't happen, first lies}&\ p_{FTF} = (1 - p)(1 - x)y \\ \text{Event doesn't happen, second lies}&\ p_{FFT} = (1 - p)x(1 - y) \\ \text{Event doesn't happen, both tell truth}&\ p_{FFF} = (1 - p)xy \end{align*}

Now if we ask what is $p\left(\text{truth}|\text{agree}\right)$, we have:

\begin{align*} \frac{p_{TTT} + p_{FFF}}{p_{TTT} + p_{FFF} + p_{TFF} + p_{FTT}} =&\ \frac{pxy + (1 - p)xy}{pxy + (1 - p)xy + p(1 - x)(1 - y) + (1 - p)(1 - x)(1 - y)} \\ =&\ \frac{xy}{xy + (1 - x)(1 - y)} \end{align*}

This is the correct result because the probability of accepting two people's judgement should not depend on the probability of the event actually being true!

And the above should be somewhat obvious because two people lying or not is independent of the probability that something happens (remember, they tell the truth by saying it did or didn't happen). Does that sound right? If something is extremely likely to happen, and they agree it did, shouldn't I highly suspect they're telling the truth?

This is what was found in the previous answer but I think needs clarification. In this case we are not asking whether or not they are telling the truth, here we are only interested in cases where they agree it did happen and there are only two: $p_{TTT}$ and $p_{FTT}$, and in only one case, $p_{TTT}$, did it actually happen:

$$ p\left(\text{happened}|\text{both say it happened}\right) = \frac{pxy}{pxy + (1 - p)(1 - x)(1 - y)} $$

And indeed, in this case, if $p = 1$, this probability is $1$ (if it always happens, you don't really need to ask), likewise if $p = 0$, this probability is $0$. Then if it's a coin flip, $p = 0.5$, it's the probability that they both tell the truth. Similarly we could find:

$$ p\left(\text{didn't happen}|\text{both say it didn't happen}\right) = \frac{(1-p)xy}{(1 - p)xy + p(1 - x)(1 - y)} $$

Or we could ask, if they agree, what's the probability that the thing actually happened? Well there are four ways to agree and two ways to agree and the event happened: told the truth and it happened or lied and it happened:

\begin{align*} p\left(\text{happened}|\text{they agreed}\right) =&\ \frac{pxy + p(1 - x)(1 - y)}{xy + (1 - x)(1 - y)}\\ =&\ p\frac{xy + (1 - x)(1 - y)}{xy + (1 - x)(1 - y)} \\ =&\ p \end{align*}

In other words the probability of the event happening is independent of whether or not they agree (which it should be). We can reverse these questions:

What's the probability they agree the event happened, if it did?

\begin{align*} p\left(\text{agreed it happened} | \text{happened}\right) =&\ \frac{pxy}{p} = xy \end{align*}

Not surprising, if it happened, the only way they agree it did is if they both tell the truth.

What's the probability that they agree it didn't happen, if it didn't happen?

$$ p\left(\text{agreed didn't happen}|\text{didn't happen}\right) = \frac{(1 - p)xy}{1 - p} = xy $$

Again, not surprising, they must tell the truth (agree it didn't happen).

And then finally, given the event happened, what's the probability that they agree?

\begin{align*} p\left(\text{agreed}|\text{happened}\right) =&\ \frac{pxy + p(1 - x)(1 - y)}{p}\\ =&\ xy + (1 - x)(1 - y) \end{align*}

So the probability that they agree is independent of the probability of the event happening and is simply the probability that they both tell the truth or they both lie. We could obviously do the same thing for "given didn't happen" and the result will be identical (for one because we could just reverse the meaning of "happened" and then $p$ just takes on the value of $1 - p$).

What if they don't know whether or not they're telling the truth?

This is outside the scope of your question but it does seem to be relevant. This is much more complicated because now we have $5$ probabilities: $p$, $x$, $p_x$, $y$, $p_y$ (where $p_x$ is the probability they're "belief" of truth actually is the truth), meaning truth table is now of size $32$ (rather than just $8$ as above). But we can use our original result to expedite our analysis. We essentially have four situations:

\begin{align*} \text{1. } & \text{Both know the truth} & p_xp_y && \text{Results in above truth table} \\ \text{2. } & \text{Both are mistaken} & \left(1 - p_x\right)\left(1 - p_y\right) && \text{"Truth'' is a lie: } (x,y) = (1 - x, 1 - y)\\ \text{3. } & \text{First mistaken} & \left(1 - p_x\right)p_y && (x, y) = (1 - x, y) \\ \text{4. } & \text{Second mistaken} & p_x\left(1 - p_y\right) && (x,y) = (x, 1 - y) \end{align*}

So now we ask what's the probability they tell the truth (truth here meaning event did or didn't happen) if they agree? This is different: originally if they both told the truth, it did (or didn't) happen; now if they tell the truth there's a possibility that they're mistaken and it didn't (or did) happen.

There are four ways they can they can agree and "tell the truth":

• Neither are mistaken ($p_xp_y$) and both tell the truth ($xy$)
• Both are mistaken and both lie: $(1 - p_x)(1 - p_y)(1 - x)(1 - y)$
• $x$ is mistaken, lies, $y$ tells truth: $(1 - p_x)p_y(1 - x)y$
• $y$ is mistaken, lies, and $x$ tells truth: $p_x(1 - p_y)x(1 - y)$

Notice (and below) that when $p_x = p_y = 1$, we are left with $\frac{xy}{xy + (1 - x)(1 - y)}$--our original result (since we originally assumed $p_x = p_y = 1$).

There are four more ways for them to agree:

• Neither are mistaken and both lie: $p_xp_y(1 - x)(1 - y)$
• Both are mistaken and both tell truth: $(1 - p_x)(1 - p_y)xy$
• $x$ is mistaken, tells truth, $y$ lies: $(1 - p_x)p_yx(1 - y)$
• $y$ is mistaken, tells truth, $x$ lies: $p_x(1 - p_y)(1 - x)y$

For what it's worth, if you really still don't believe $p$ shouldn't be involved, the above "eight" lines represent $16$ (from $2 \times 4 + 2 \times 4$) different cases out of a total of $32$ if we really still need to include $p$ in the truth table. Each line represents the sum of its probability times $p$ then $1 - p$ and added. Either the event happened or didn't and they "agreed" or "didn't" based on truth and probability of knowing the truth.

This gives a very complicated expression for the probability that they are "telling the truth":

$$ p\left(\text{truth}|\text{agreed}\right) = \frac{p_{correct}}{p_{correct} + p_{wrong}} $$

Where

\begin{align*} p_{correct} =&\ p_xp_yxy + (1 - p_x)(1 - p_y)(1 - x)(1 - y) \\ &\ + (1 - p_x)p_y(1 - x)y + p_x(1 - p_y)x(1 - y) \end{align*}

and

\begin{align*} p_{wrong} =&\ p_xp_y(1 - x)(1 - y) + (1 - px)(1 - p_y)xy\\ & + (1 - p_x)p_yx(1 - y) + p_x(1 - p_y)(1 - x)y \end{align*}

This looks complicated. What I expect is that the probability that a person tells the truth is the probability that they know the truth and tell the truth plus the probability that they are mistaken and lie: $p_xx + (1 - p_x)(1 - x)$. Thus from our previous result we could have immediately written down the answer for the probability that they tell the truth given that they agree:

$$ p\left(\text{truth}|\text{agree}\right) = \frac{xy}{xy + (1 - x)(1 - y)} $$

Where

\begin{align*} x =&\ p_xx + (1 - p_x)(1 - x) \\ y =&\ p_yy + (1 - p_y)(1 - y) \end{align*}

This leads to a similarly complicated expression:

Where the numerator is:

$$ num = (p_xx + (1 - p_x)(1 - x))(p_yy + (1 - p_y)(1 - y)) $$

And Denominator is:

\begin{align*} denom = &\ num + (1 - (p_xx + (1 - p_x)(1 - x))(1 - (p_yy + (1 - p_y)(1 - y)) \end{align*}

You should be able to show that these two expressions are equivalent. Specifically I checked at Wolfram (in my query $w = p_x$ and $z = p_y$--w is next to x and z is next to y):

1. Denominator of first expression (using truth table)
2. Denominator of second expression (using already found formula for truth given they agree)

And since I did plug it in: Here is the result from Wolfram for the full fraction--there are some interesting results in Wolfram's analysis. You can see the values of 4, 8, 16 showing up in some of its decompositions--you also see the different permutations we expect to find being grouped together.