How can I evaluate this limit? Give me a hint, please.
$$\lim_{n \to\infty}\frac{1\cdot2\cdot3+2\cdot3\cdot4+\dots+n(n+1)(n+2)}{\left(1^2+2^2+3^2+\dots+n^2\right)^2}$$
Note that $$(1^2+2^2+\ldots+n^2)^2\ge 1^4+2^4+\ldots+n^4$$ Now we can use Stolz–Cesàro theorem (kind of a discrete l'Hôpital's rule): $$\lim_{n\to\infty}\frac{1\cdot2\cdot3+2\cdot3\cdot4+\ldots+n(n+1)(n+2)}{1^4+2^4+\ldots+n^4}=\lim_{n\to\infty}\frac{n(n+1)(n+2)}{n^4}=0$$ And conclude the original limit was also $0$.
The numerator is bounded above by $n.n(n+1)(n+2)$, fourth degree. The denominator is exactly $(2n^3+3n^2+n)^2/36$ (from the square pyramidal number formula), sixth degree. So the denominator "wins".
Hint: $\frac{n^p}{n^q} \rightarrow 0$ if $p<q$ (something small divided by something big) and $\frac{n^p}{n^q} \rightarrow \infty$ if $p>q$ (something big divided by something small).
Which one of these two scenarios is most like your problem?
Hint:
$$1\cdot 2\cdot 3 + \dots +n(n+1)(n+2) \le n(n+2)^3 \le 8n^4$$ $$(1^2 + 2^2 + \dots + n^2)^2 = \left(\frac{n(n+1)(2n+1)}{6}\right)^2\ge \frac{1}{9}n^6$$ Hence
$$\frac{1\cdot 2\cdot 3 + \dots +n(n+1)(n+2)}{(1^2 + 2^2 + \dots + n^2)^2} \le \frac{8n^4}{1/9\cdot n^6}=\frac{72}{n^2}\to 0$$
$$ \lim_{n\to \infty} \frac{\sum\limits_{k=1}^n k(k+1)(k+2)}{\left[\sum\limits_{k=1}^n k^2\right]^2}= \lim_{n\to \infty} \frac{\sum\limits_{k=1}^n k^3+3\sum\limits_{k=1}^n k^2+2\sum\limits_{k=1}^n k}{\left[\sum\limits_{k=1}^n k^2\right]^2} = \lim_{n\to \infty} \frac{\frac{n^2(n+1)^2}{4}+ \frac{n(n+1)(2n+1)}{2}+n(n+1)}{\left[\frac{n(n+1)(2n+1)}{6}\right]^2} = \lim_{n\to \infty} \frac{\frac{1}{4}+ \frac{2n+1}{2n(n+1)}+\frac{1}{n(n+1)}}{\frac{(2n+1)^2}{6^2}}= \lim_{n\to \infty} \frac{36\left(\frac{1}{4}+ \frac{1}{n+1}+\frac{1}{2n(n+1)}+\frac{1}{n(n+1)}\right)}{(2n+1)^2} =\frac{36\left(\frac{1}{4}+\frac{1}{\infty}+\frac{1}{\infty}+\frac{1}{\infty}\right)}{\infty}= 0 $$