I tried root and ratio tests but it didn't work. Also, i can't use integral tests (and other "uncommon" ones) in this homework.
(Prove that the series is convergent) $$\sum_{n=1}^{\infty}\left(\frac{n}{n+1}\right)^{n^2} = \frac{1}{2} + \left(\frac{2}{3}\right)^4 + \left(\frac{3}{4}\right)^9 + ...$$
The Bernoulli Inequality says that for $n\ge1$, $$ \left(1+\frac1n\right)^n\ge1+\frac{n}{n}=2 $$ Thus, $$ \left(\frac{n}{n+1}\right)^{\large n^2}\le\left(\frac12\right)^n $$ Thus, we can compare this to a geometric series.
Even if you don't know many limits, note that by expanding $(1+{1\over n})^n$ and dropping all terms except the two lowest order terms, one has $(1+{1\over n})^n\ge 1+n\cdot{1\over n}=2$. Thus $$0\le\Bigl({n\over n+1}\Bigr)^{n^2}={1\over\bigl[ \,(1+{1\over n})^n\,\bigr]^n}\le {1\over 2^n}.$$ The convergence of your series now follows by comparison to the convergent geometric series $\sum\limits_{n=1}^\infty (1/2)^n$.
(Using the "well-known" limit $\lim\limits_{n\rightarrow\infty}(1+{1\over n})^n=e$, you can conclude that the terms $(1+{1\over n})^n$ eventually exceed $2$; this will suffice for comparing your series with $\sum\limits_{n=1}^\infty (1/2)^n$. Of course, if you do make use of this limit, using the Root Test is somewhat easier.)
The root test should work.
Notice the following equivalencies:
$$\left(\frac{n}{n+1}\right)^{n^2}=\left[\left(\frac{n}{n+1}\right)^n\right]^n= \frac{1}{\left[\left( 1+{1\over n}\right)^n\right]^n}$$
The right-hand side should look very familiar. You can certainly use the limit comparison test to prove convergence of your series.
We can interpret the series as power series evaluated at 1. Using the Hadamard formula we obtain $1/R = \lim_{n \rightarrow \infty} ((\frac{n}{n+1})^{n^2})^\frac{1}{n} = \lim_{n \rightarrow \infty} (1+\frac{-1}{n+1})^{n+1} \frac{n+1}{n} = e^{-1}$ so $R = e > 1$ and the series converges.
