From the summation expression I assume $N \in \mathbb{N}$, and from your attempt at $P'(N)$ I assume you are trying to maximise w.r.t. $N$. If this is the case, then there is an alternative to finding turning points of the Gamma involved expression.
If $a>0$ is fixed and we are trying to find $N^*\in \mathbb{N}$ which maximises $P(N)$, we have:
$$P(N)=\prod_{k=1}^N\frac{k^a}{a^k} = \frac{(N!)^a}{a^{\frac{N(N+1)}{2}}}$$
$$\text{Let }r(N) = \frac{P(N+1)}{P(N)}= \frac{(N+1)^a}{a^{N+1}}$$
This ratio tells us whether at $N$ the function is increasing ($r>1$) or decreasing ($r < 1$). So we look at when
$r > 1 \iff (N+1)^a > a^{N+1} \iff \sqrt[N+1]{N+1} > \sqrt[a]{a}$.
The function $f(x) = \sqrt[x]{x}$ has its maximum at $x=e$ and is decreasing on both sides. So eventually $r(N)< 1$ for all large $N$. Looking at where $a$ falls, we can use this to find the turning points of $r(N)$ and hence the maximum of $P(N)$.
For e.g. if $a > e$, we have $N^*$ could be $1, 2$ or the first integer larger than $a$.