On the convergence of moments under weak convergence

Question

It is known that the collection of finite mixtures of Gaussian Distributions over $\mathbb{R}$ is dense in $\mathcal{P}(\mathbb{R})$ (the space of probability distributions) under convergence in distribution metric.

I'm interested to know the following:

Let $P_X$ be a random variable with finite $p$ th moment i.e. $\mathbb{E}_{P_X}[|X|^p]<\infty$, and $P_{X_n}\stackrel{d}{\to} P_X$ where $P_{X_n}$ are mixtures of Gaussian distributions. Then, suppose $X_n \sim P_{X_n}$ and $X\sim P_X$, does it follow that $$\mathbb{E}[|X_n|^p] \to \mathbb{E}[|X|^p]$$? $\quad (*)$

My attempt:

I've been able to show that $$\liminf_{n \to \infty} \mathbb{E}[|X_n|^p] \geq \mathbb{E}[|X|^p]$$ In fact this didn't even use the mixtures part. However I'm having difficulty showing the upper bound. Here are some steps: \begin{eqnarray} \mathbb{E}[|X_n|^p] &=& \mathbb{E}[|X_n|^p1_{\{|X_n|\leq A\}}] + \mathbb{E}[|X_n|^p1_{\{|X_n|>A\}}] \\ &=& \mathbb{E}[|X_n|^p1_{\{|X_n|\leq A\}}] + A^pPr[|X_n|\geq A] + \mathbb{E}[(|X_n|^p-A^p)1_{\{|X_n|>A\}}] \end{eqnarray} Let $f(x)=x1_{0\leq x\le A} + A1_{x\geq A}$ which is a continuous and bounded function. Then the first two terms of RHS are equal to $\mathbb{E}[f(|X_n|^p)]$, which by definition of weak convergence, will converge to $\mathbb{E}[f(|X|^p)]$ as $n \to \infty$. Now we may use MCT as $A\to \infty$ to get $\mathbb{E}[|X|^p]$. Hence in a nutshell, we need to show: \begin{equation} \limsup_{A\to\infty} \limsup_{n \to \infty} \mathbb{E}[(|X_n|^p-A^p)1_{\{|X_n|>A\}}] \leq 0 \end{equation} However I do not know how to proceed from here.

I'd like to point out that I don't know the answer to the question I asked in $(*)$ but my guess is that this is true. The reason is because I read a similar result in Soren Asmussen's book. The result states that for distributions over non-negative reals, we have phase type distributions not only being weak dense but also the moments converging. But in the provided proof, they said "It can be easily shown that the moments converge".

Update: Sincerest apologies for I forgot one very essential condition. The mixtures are not any mixtures but specific ones. Namely, I'm looking at distributions $P_X$ such that $P_{X_n}$ has the form $\sum_{k=1}^n \alpha_k \mathcal{N}(\mu_k;P_k)$ ($\mathcal{N}(\mu_k;P_k)$ stands for Gaussian distribution with mean $\mu_k$ and variance $P_k$.) where given $P<\infty$, $\sum_{k=1}^n \alpha_k =1$, $\alpha_k >0$ and $\sum_{k=1}^n\alpha_k P_k = P$.

Update 2: The reference I mentioned earlier is Soren Asmussen "Applied probability and queues", 2nd edition, page 84.

Update 3: Looks like I misinterpreted Asmussen. What he wanted to say was that for any distribution $P_X$ with finite $p$th moment, there exists a sequence of phase type distributions $P_X^k$ such that $P_X^k\stackrel{d}{\to}P$ and $E_{P_X^K}[|X|^p] \to E_{P_X}[|X|^p]$. This doesn't mean any weak converging mixture will have moment convergence as the answer points out.

Answer 1

False, in general, as @PhoemueX said.

Here is a counterexample. Let $X\sim N(0,1)$ be Gaussian. To construct the distribution of $X_n$ in its very essential peculiar form, let $a_1=a_2=\dots = a_n=1/n,$ let $\mu_k = 0$ for all $k<n$ and let $\mu_n=n^{1/p},$ and let $P_k = (n/(n-1))(1-1/(nn!))$ for $k<n$, and $P_n=1/n!$. Then let $X_n$ have distribution $\sum_{k=1}^na_kN(\mu_k,P_k)$. Then $X_n$ converges in distribution to $N(0,1)$ but $|EX_n|^p$ converges, but not to $E|X|^p$, but rather to $1+E|X|^p$.

The intuition is that $X_n$ is most of the time very nearly $N(0,1)$ but very occasionally something else, that throws the expectation off from what the OP expects. The restrictions of form of allowed densities (mixtures of point masses, mixtures of gaussians with overall variance constraints, etc) that the OP is interested in do no good in enforcing the uniform integrability constraints that are needed to defeat the problem brought up by @PhoemueX.

Answer 2

Please note the following result (a similar result is mentioned also in Asmussen's book Theorem 4.2): Suppose that $F_k\stackrel{w}{\to}F$ and $\sup_k\mathsf{E}_{F_k}[|X|^q]<\infty$, then $$\lim_k\mathsf{E}_{F_k}[|X|^p]=\mathsf{E}_F[|X|^p]\qquad\text{for all }p<q.$$ This results could be prove by the following estimation: $$ \sup_k\mathsf{E}_{F_k}[|X|^p1_{|X|>A}]\le \frac1{A^{q-p}}\sup_k\mathsf{E}_{F_k}[|X|^q]\to 0 \qquad \text{as }A\to\infty.$$ Using above results to sequence of mixed Gaussian distributions $$ F_n=\sum_{k=1}^n\alpha_k^{(n)}\mathcal{N}(\mu_k^{(n)},P_k^{(n)}), \quad \alpha_k^{(n)}\ge 0, \sum_{i=1}^n\alpha_k^{(n)}=1. $$ we have following consequence: Suppose that $F_k\stackrel{w}{\to}F$ and $$ \sup_n\biggl(\sum_{k=1}^n\alpha^{(n)}_k[(\mu^{(n)}_k)^2+P^{(n)}_k]^{q/2}\biggl)<\infty. $$ then
$$\lim_k\mathsf{E}_{F_k}[|X|^p]=\mathsf{E}_F[|X|^p]\qquad \text{for all }p<q.$$ In particular, if $$ \sup_{k,n}[|\mu_k^{(n)}|+P_k^{(n)}]<\infty, $$ then $$\lim_k\mathsf{E}_{F_k}[|X|^p]=\mathsf{E}_F[|X|^p]\qquad \text{for all }p>0.$$ At last, thanks Phoemue and Kumchi's example, it explain that to guarantee the convergence of moments some additional conditions are unavoidable.