You can show that $\operatorname{ad}(y) \operatorname{ad}(x)$ is nilpotent when $x$ lies in the nilradical. Write down a high power of $\operatorname{ad}(y) \operatorname{ad}(x)$:
$$(\operatorname{ad}(y) \operatorname{ad}(x))^n = \operatorname{ad}(y) \operatorname{ad}(x) \cdots \operatorname{ad}(y) \operatorname{ad}(x)\operatorname{ad}(y) \operatorname{ad}(x) \,.$$
Now we use that $\operatorname{ad}(x)\operatorname{ad}(y) - \operatorname{ad}(y)\operatorname{ad}(x) = \operatorname{ad}([x, y])$ in order to move the right-most factor $\operatorname{ad}(y)$ one position to the left:
$$\begin{align*}
&\operatorname{ad}(y) \operatorname{ad}(x) \cdots \operatorname{ad}(y) \operatorname{ad}(x) \operatorname{ad}(y) \color{red}{ \operatorname{ad}(x) \operatorname{ad}(y)} \operatorname{ad}(x) \\
& \quad = \operatorname{ad}(y) \operatorname{ad}(x) \cdots \operatorname{ad}(y) \operatorname{ad}(x) \operatorname{ad}(y) \color{red}{\operatorname{ad}(y)\operatorname{ad}(x)} \operatorname{ad}(x) \\
& \qquad + \operatorname{ad}(y) \operatorname{ad}(x) \cdots \operatorname{ad}(y) \operatorname{ad}(x) \operatorname{ad}(y) \color{red}{\operatorname{ad}([x, y])} \operatorname{ad}(x)
\end{align*}$$
We can keep moving factors $\operatorname{ad}(y)$ to the left, until we obtain a finite sum of products of the form
$$\operatorname{ad}(y)^k \cdot (\text{product of }n\text{ factors }\operatorname{ad}(x) \text{ or }\operatorname{ad}([x,y])\text{ in any order}) \,.$$
It suffices now to show that such a product is zero when $n$ is large enough. This is because $\operatorname{ad}(\mathfrak n)^n(\mathfrak g) \subset \operatorname{ad}(\mathfrak n)^{n-1}(\mathfrak n)$ is zero when $n$ is large enough, because $\mathfrak n$ is nilpotent.
If in the argument you do not start with the right-most factors $\operatorname{ad}(y)$, you will in the end also get factors of the form $\operatorname{ad}([[x,y], y]), \operatorname{ad}([[[x,y], y], y]), \ldots$ but that is no problem as those brackets still lie in the nilradical.