An extremum on the graph is a point where the function is locally maximal or minimal, and occurs when $f'(x)=0$. A point of inflection is when the curvature changes sign, and occurs when $f''(x)=0$.
So, we know that
\begin{align}
f'(x)&=3ax^2+2bx+c,\\
f''(x)&=6ax+2b.
\end{align}
We want $f'(x)=0$ when $x=0$, and therefore we can see that $f'(x)_{x=0}=3a\cdot 0^2$ $+2b\cdot 0+c=c$, and therefore $c=0$. Also, we want $f''(x)=0$ when $x=-1$, and therefore we can see that $f''(x)_{x=-1}=-6a+2b=0$, and therefore $a=b/3$. So our function is now,
$$f(x)=ax^3+3ax^2+d.$$
Now, when $x=0$, $f(x)=3$, so we see that $d=3$, and when $x=-1$, $f(x)=1$, and so $a=-1$, which gives the function
$$f(x)=-x^3-3x^2+3.$$
However, to be more rigorous, you need to apply the first derivative test to each point under consideration to make sure that we are looking at a point of inflection, or a extremal point. Recall that the first derivative test says that: if $f(x)$ is continuous at a point $x_0$, then:
i) if $f'(x)>0$ on the left of $x_0$, and $f'(x)<0$ on the right of $x_0$, then we have a local maximum,
ii) if $f'(x)<0$ on the left of $x_0$, and $f'(x)>0$ on the right of $x_0$, then we have a local minimum,
iii) and if $f'(x)$ has the same sign on the left and the right of $x_0$, then we have a point of inflection.