I wish to use the Computational formula of the variance to calculate the variance of a normal-distributed function. For this, I need the expected value of $X$ as well as the one of $X^2$. Intuitively, I would have assumed that $E(X^2)$ is always equal to $E(X)^2$. In fact, I cannot imagine how they could be different.
Could you explain how this is possible, e.g. with an example?
Assume $X$ is a random variable that is 0 half the time and 1 half the time. Then $$EX = 0.5 \times 0 + 0.5 \times 1 = 0.5$$ so that $$(EX)^2 = 0.25,$$ whereas on the other hand $$E(X^2) = 0.5 \times 0^2 + 0.5 \times 1^2 = 0.5.$$ By the way, since $Var(X) = E[(X - \mu)^2] = \sum_x (x - \mu)^2 P(x)$, the only way the variance could ever be 0 in the discrete case is when $X$ is constant.
Let $EX=\mu$ and $E(X-\mu)^2=\sigma^2$, then
$$ EX^2 = E[X-\mu+\mu]^2=\\ =E(X-\mu)^2+2E[(X-\mu)\mu]+E(\mu^2)=\\=\sigma^2+2\mu E(X-\mu)+\mu^2=\\ =\sigma^2+\mu^2 $$
So $EX^2 =\sigma^2+\mu^2$, no matter the distribution, and $EX^2\ne(EX)^2$ unless the variance equals zero.
One is an average of squares, the other a square of an average. In general, when you reverse two procedures (mix cookies, bake cookies), you have no right to expect the same outcome.
Note that your logic applied to a uniform distribution would give that $$(x_1+x_2+\cdots+x_n)^2=n({x_1}^2+{x_2}^2+\cdots+{x_n}^2)$$ which is clearly not true in general.
Let us take for example $X$ the standard normal, or any normal with mean $0$. Then $E(X)=0$.
But $X^2$ is always positive, so clearly its mean must be positive.
This shows that (in this case) $E(X^2)\ne (E(X))^2$.
In fact, when the expectations exist, $E(X^2)>(E(X))^2$ except when $X$ is constant with probability $1$.
Say you have a fair coin that says $X=1$ on one side and $X=3$ on the other side. You flip the coin. Clearly, $E(X)=\frac12(1+3) = 2$.
If you are counting $X^2$ instead of $X$, then one side of the coin is worth $1^2=1$ and the other side is worth $3^2=9$, so $E(X^2) = \frac12(1+9)=5$.
$5\ne 2^2$.
My turn:
Let $X$ be uniformly distributed on $[0,1]$. The $E X =\int_{t=0}^1 t dt = \frac{1}{2}$, but $E X^2 =\int_{t=0}^1 t^2 dt = \frac{1}{3}$.
It's a late answer, but it might be useful for anyone reading it.
For a random variable $X$, $E(X^{2})= [E(X)]^{2}$ iff the random variable $X$ is independent of itself. This follows from the property of the expectation value operator that $E(XY)= E(X)E(Y)$ iff $X$ and $Y$ are independent random variables. In the case of a single type of random variable $X$, this statement reduces down to the statement that the random variable is independent of itself.
Statement: Two random variables are independent iff:
$\mathrm{F}_{XY}(x,y)= \mathrm{F}_{x}(x)\mathrm{F}_{y}(y); $ for all $X= x$, $Y=y$
where $\mathrm{F}_{X}(x)$, $X= x$ denotes the cumulative distribution function of an arbitrary random variable $X$. Now as this answer also suggests, a random variable can be independent of itself iff it realizes a fixed (constant) value. This follows from:
$\mathrm{F}_{XX}(x,x)=\mathrm{F}_{X}(x)=\mathrm{F}_{X}(x)\mathrm{F}_{X}(x)$
and $\mathrm{F}_{X}(x)\in$ {$0,1$}, for any $X= x$. The only way for the cumulative distribution function to realize the value $1$ for any $X= x$ is for random variable $X$ to be a constant, i.e., $X= x_{0}$, where $x_{0}$ is the (only) element of the (possibly only) non-empty subset (i.e. a singleton) of the $\sigma$-algebra of the set $E$ (not the expectation value), which is the set of all of values that the random variable $X$ can realize (a more rigorous proof for this last step may require a measure-theoretic approach).
Notice that $E(X^{2})= [E(X)]^{2}$ also implies $E(X^{2})-[E(X)]^{2}=0 \Rightarrow Var(X)=0$, which is always satisfied for a constant random variable $X$. This can also give an intuition for the reasoning above.
Therefore, $E(X^{2})\neq [E(X)]^{2}$ in general.
May as well chime in :)
Expectations are linear pretty much by definition, so $E(aX + b) = aE(X) + b$. Also linear is the function $f(x) = ax$. If we take a look at $f(x^2)$, we get
$f(x^2) = a(x^2) \not= (ax)^2 = f(x)^2$.
If $E(X^2) = E(X)^2$, then $E(X)$ could not be linear, which is a contradiction of its definition. So, it's not true :)
Assuming $X$ is a discrete random variable $E(X)=\sum x_ip_i$. Therefore $E(X^2)=\sum x_i^2p_i$ while $[E(X)]^2=\left(\sum x_ip_i\right)^2$. Now, as Robert Mastragostino says, this would imply that $(x+y+z+\cdots)^2=x^2+y^2+z^2+\cdots$ which is not true unless $X$ is constant.
