Consider evaluation of sums of following form:
$$S_p\stackrel{def}{=}\sum_{k=1}^n\lfloor\sqrt[p]{k}\rfloor$$
where $p$ is a positive integer $\ge 2$. In the special case $p = 2$, this reduces to the sum we want to calculate.
Let $a = \lfloor \sqrt[p]{n} \rfloor$ and take a sufficiently small $\epsilon > 0$ such that
$$\lfloor \sqrt[p]{k} + \epsilon \rfloor = \lfloor \sqrt[p]{k} \rfloor\quad\text{ for all } k = 1, 2, \ldots, n$$
The introduction of the $\epsilon$ piece make the function $\lfloor \sqrt[p]{x} + \epsilon\rfloor$ continuous at the jumps of $\lfloor x \rfloor$. This allows us to rewrite $\mathcal{S}_p$ as a Riemann-Stieljes integral.
$$
\mathcal{S}_p = \sum_{k=1}^n \lfloor \sqrt[p]{k} \rfloor
= \sum_{k=1}^n \lfloor \sqrt[p]{k} + \epsilon \rfloor
= \int_{1^-}^{n^+} \lfloor \sqrt[p]{x} + \epsilon \rfloor d \lfloor x \rfloor
$$
Integrate by part, we get
$$
\mathcal{S}_p
= \bigg[ \lfloor \sqrt[p]{x} + \epsilon \rfloor \lfloor x \rfloor \bigg]_{x=1-}^{n^+}
- \int_{1^-}^{n^+} \lfloor x \rfloor d \lfloor \sqrt[p]{x} + \epsilon \rfloor
= na
- \int_{1+\epsilon}^{\sqrt[p]{n}+\epsilon} \lfloor (s-\epsilon)^p \rfloor d \lfloor s \rfloor
$$
Convert the last integral back into a sum, we find
$$
\mathcal{S}_p
= na - \sum_{s=2}^a (s^p - 1)
= (n+1)a - \sum_{s=1}^a s^p
= (n+1)a - \frac{B_{p+1}(a+1)-B_{p+1}(0)}{p+1}
$$
where $B_k(x)$ is the Bernoulli polynomial of order $k$
For the special case $p = 2$, the sum we want is
$$
\bbox[8pt,border: 1px solid blue;]{
\sum_{k=1}^n \lfloor\sqrt{k}\rfloor = \mathcal{S}_2 =
(n+1) a - \frac16 a(a+1)(2a+1),\quad a = \lfloor\sqrt{n}\rfloor
}
$$
This is equivalent to the formula in Markus Scheuer's answer.
For the case $p = 3, 4, 5$. this leads to
$$\begin{align}
\sum_{k=1}^n \lfloor\sqrt[3]{k}\rfloor = \mathcal{S}_3 & =
(n+1) a - \frac14\left((a+1)^4 - 2(a+1)^3 + (a+1)^2\right)\\
\sum_{k=1}^n \lfloor\sqrt[4]{k}\rfloor = \mathcal{S}_4 & =
(n+1) a - \frac15\left(
(a+1)^5 - \frac52(a+1)^4 + \frac53 (a+1)^3 - \frac16(a+1)
\right)\\
\sum_{k=1}^n \lfloor\sqrt[5]{k}\rfloor = \mathcal{S}_5 & =
(n+1) a - \frac16\left(
(a+1)^6-3(a+1)^5 + \frac52 (a+1)^4 - \frac12 (a+1)^2
\right)\\
\end{align}
$$
reproducing the formulas in Claude Leibovici's answer.