This is a problem with many parts, and it's difficult to tell from the question
how much detail you need in each part.
The basic idea is that to write a formula that would tell you what the
final value of the account would be, given the starting value,
contribution at the end of each month, rate of interest earned per month,
and number of months.
The formula is given in https://math.stackexchange.com/a/395799;
for payments made at the end of each month it is
$$P=B(1+i)^n+A\dfrac{(1+i)^n-1}{i}$$
where $B$ is the initial value, $A$ is the deposit each month,
$i$ is the interest rate, and $P$ is the final value of the account.
You are given the values of all the quantities in this formula except for $i$,
which is the variable you must solve for.
Unfortunately, there is no closed-form solution to find $i$,
so instead you must rely on some kind of guess-and-refine method
such as Newton's method.
A good first guess for $i$ for most purposes would be to simply
add up all the payments (which in your example comes to $4600$)
and subtract this from the final value to find out how much interest
the account had to earn, which in this example is $400$.
Then estimate the average value of the account during the entire time;
it's difficult to do this exactly without knowing the interest rate,
but for the purpose of the first guess, try averaging the starting
and ending values: $\frac12(1000 + 5000) = 3000$.
Now solve for $i$ in $3000 (1 + 36i) = 400$.
(The factor of $36$ is because we're looking for monthly interest,
and there are $36$ months between the initial and final values of this account.)
And remember, this is only the first guess;
you must apply some further steps to refine the value
until the formula above produces the correct value of $P$
(to within a reasonable degree of accuracy) given the other quantities.
