By definition $$E\{X\}=\sum_{x}x\cdot \Pr\{X=x\}$$since the $x$s over which the sum is taken and calculated are in $C$ and $\Pr\{X=x\}$ are some coefficients that sum up to $1$ and because of convexity (that yields to the fact that is any bunch of points lie in a convex set, then so does any convex combination of them, i.e. points that are a linear combination of the original points with non-negative coefficients summing up to $1$) and closedness (needed for continuous PDFs) of $C$, we conclude that $E\{X\}\in C$.
Intuition
The intuition behind this proof is that if a cloud lies completely in a convex set (by cloud we mean the set of points that are probable within a PDF) , then so does the middle point (mathematical expectation) of that cloud, even if the cloud is not convex itself.
Examples for a probabilistic cloud:
If we define a PDF within which all the points in a 3D unit sphere are equiprobable, then the cloud is that unit sphere with its intensity being equal in all points.
For the standard Gaussian distribution over $\Bbb R^3$, the cloud becomes whole the $\Bbb R^3$ mostly intensified around the origin.
Define $$p(X)=\begin{cases}e^{-x_1-x_2-x_3}&,\quad x_i\ge 0\\0&,\quad \text{Otherwise}\end{cases}$$then the cloud becomes $\Bbb (R^{\ge 0})^3$.