What is the smallest and largest possible values for the variance?

Question

Suppose $P( X \in \{1,2,3\}) = 1$ and $E(X) =2.5.$

What is the smallest and largest possible values for the variance?

My understand: So what I understand is variance finds the distance between each element and the mean. So the closer 2 is to E(X) the farther 1 and 3 can be from E(X). But I have no clue how to obtain this..

Answer 1

Let the probabilities be $P(X = x_i)$ be $p_i$ for values of $i$ in $\{1,2,3\}$

Then

$p_1 + p_2 + p_3 = 1$

$p_1 + 2p_2 + 3p_3 = 2.5$ (from $E(X) = 2.5$)

and we need to maximise/minimise $p_1 +4p_2 + 9p_3 - 6.25$ (from $V(X) = E(X^2) - \{E(X)\}^2$)

Hope this gives you a start. This is a linear problem, so that the min and max can be reached at the boundary points

This problem can be solved more intuitively without the use of all these symbols and constraints.

The intuition, as outlined by Henry in his comment comes from

(1) Maximum Variance -- at end points, so put $p_2 = 0$, then $p_1 = \frac{1}{4}$ and $p_3 = \frac{3}{4}$ so $\color{blue}{V_{max} = 0.75}$

and

(2) Minimum variance -- have data points near mean, so $p_1 = 0$ and $p_2 = p_3 = \frac{1}{2}$ so $\color{blue}{V_{min} = 0.25}$

Answer 2

Although the two things are equal, I think it is easier to use $$\operatorname{Var}(X) = \mathbb{E}\Big[(X-\mathbb{E}X)^2\Big]$$ rather than $\operatorname{Var}(X)=\mathbb{E}X^2-(\mathbb{E}X)^2$.

Let $\mathbb{P}[X=i]=p_i$, then

\begin{align} \operatorname{Var}(X) &= \mathbb{E}\Big[(X-\mathbb{E}X)^2\Big] \\ &= \sum_{i \in \{1,2,3\}}(i-2.5)^2 \cdot p_i \\ &= (-1.5)^2\cdot p_1 + (-0.5)^2 \cdot p_2 + (0.5)^2 p_3 \\ &= 2.25\cdot p_1 + 0.25\cdot p_2 + 0.25\cdot p_3. \tag{$\spadesuit$} \end{align}

Maximizing and minimizing this under constraints $p_1+p_2+p_3=1$ and $1\cdot p_1 + 2\cdot p_2 + 3\cdot p_3 = 2.5$ is simple, because the weights of $p_2$ and $p_3$ are equal and these are both $0.5$ away from the mean:

• To minimize variance move all the probability mass away from $p_1$ –distribute everything between $p_2$ and $p_3$ in a way that gives you the correct expected value.
• To maximize variance move the probability mass as much as possible to $p_1$. To achieve the correct expected value you have to have some counterbalance in $p_3$, but $p_2$ should be equal to zero.

Taking into account the two bullet points, the formula marked by $(\spadesuit)$ has a nice intuition: to minimize variance, move the probability mass as much as possible in the direction of the mean. To maximize the variance do the opposite – move the probability mass away from the mean.

I hope this helps $\ddot\smile$

Answer 3

Let $P(X=i)=p_i$ where $i=1,2,3$ and $p_1+p_2+p_3=1$.

$E(X)=2.5$ gives $p_1+2p_2+3p_3=2.5$.

Further, $Var(X)=2.25p_1+0.25p_2+0.25p_3=2.25p_1+0.25(1-p_1)=2p_1+0.25$

Now notice that $p_2+2p_3=1.5$ from subtracting the first two equations hence $p_2=1.5-2p_3$. This shows $p_1=1-p_2-p_3=1-1.5+2p_3-p_3=-0.5+p_3$

So the feasible region is $(p_1,p_2,p_3)=(p_3-0.5,1.5-2p_3,p_3)$

Now since $p_i\in(0,1)$ for all $i$, $p_3-0.5>0$ implies $p_3>0.5$ further $1.5-2p_3>0$ implies $p_3<0.75$ and $1.5-2p_3<1$ implies $p_3>0.25$. So the range of $p_3$ is $(0.5,0.75)$.

Now writing $p_1$ in terms of $p_3$ in the variance form, $2p_1+0.25=2p_3-1+0.25=2p_3-0.75$.

This is therefore maximum for $p_3=0.75$ i.e. $Var_{max}=2\times 0.75-0.75=0.75$ and this is minimum for $p_3=0.5$ i.e. $Var_{min}=2\times0.5-0.75=0.25$.

There you go!

Answer 4

Let $p=\mathsf P(X\in\{1\}), q=\mathsf P(X\in\{2\})$

$\begin{align}2.5 ~=~& \mathsf E(X) \\ =~& p+2q+3(1-p-q)\\ =~& 3-2p-q \\ \therefore~q~=~&0.5-2p\end{align}$

Thus $0\leq p\leq 1, 0\leq 0.5 -2p\leq 1$, and $0\leq 1-p-0.5+2p\leq 1$, meaning

$$\therefore~~0\leq p\leq 0.25$$

Now use that and $\mathsf {Var}(X)= p+4q+9(1-p-q)-(2.5)^2 = 2p+0.25$ to find the minimum and maximum variance.