This problem essentially wants you to find bases for $\mathbb R^5$ and $\mathbb R^4$ such that the matrix of $f$ has the desired form. Recalling that the columns of a transformation matrix are the images of the domain basis vectors, we can see that the last few columns get mapped to zero, so the corresponding basis vectors of $\mathbb R^5$ must span $\ker f$. Similarly, the leading columns tell us that the images of the other basis vectors are a basis for $\operatorname{im}f$.
So, start by computing a basis for the kernel (null space) of $f$ and extend that to a complete basis of $\mathbb R^5$. $T$ will be the inverse of the matrix with these vectors as its columns. For $S$, compute $Mv$ for each vector $v$ of the basis extension that you just computed. This will be a basis for the image of $f$. Extend that to cover all of $\mathbb R^4$. Again, $S$ will be the inverse of the matrix with these vectors as its columns.
Note that the matrices $S$ and $T$ aren’t unique. You can freely choose any basis for the kernel, any extension of it, and any extension of the image basis.
You can proceed row-reducing $M$: $$\begin{bmatrix}-2&0&0&0&-2\\0&1&-2&0&0\\-2&0&0&1&1\\0&1&-2&1&3\end{bmatrix} \to
\begin{bmatrix}1&0&0&0&1\\0&1&-2&0&0\\0&0&0&1&3\\0&0&0&0&0\end{bmatrix}.$$ The rref gives you almost everything you need for the solution. You can read a basis for the null space of $M$ from the rref: $(0,2,1,0,0)$ and $(1,0,0,3,-1)$. The non-zero rows of the rref also give you a basis for the row space, which is the orthogonal complement of the null space. Right-multiplying $M$ by the matrix with these vectors as its columns gives $$\begin{bmatrix}-2&0&0&0&-2\\0&1&-2&0&0\\-2&0&0&1&1\\0&1&-2&1&3\end{bmatrix}\begin{bmatrix}1&0&0&1&0\\0&1&0&0&-2\\0&-2&0&0&-1\\0&0&1&3&0\\1&0&3&-1&0\end{bmatrix} = \begin{bmatrix}-4&0&-6&0&0\\0&5&0&0&0\\-1&0&4&0&0\\3&5&10&0&0\end{bmatrix}.$$ The columns of this matrix are the images of the domain basis vectors. The non-zero columns are a basis for the image of $M$, so take them and extend to a complete basis of the codomain with any linearly-independent vector: $(0,0,0,1)^T$ will do here. Collecting these vectors into a matrix and left-multiplying by its inverse will produce the required matrix. The problems want $SMT^{-1}$, though, so $S$ and $T$ will be the inverses of the two matrices that you’ve constructed.
You could also proceed as in Omnomnomnom’s now-deleted answer. Observe that the first, second and fourth columns of the rref of $M$ already have the required form. Right-multiplying a matrix by the $j$th column of the identity (i.e., the $j$th standard basis vector) picks out its $j$th column, so start building the right-hand matrix by taking $\mathbf e_1$, $\mathbf e_2$ and $\mathbf e_4$ as its first three columns. The rref also tells you that the third column of $M$ is $-2$ times the second and the last column is a linear combination of the first and fourth. Since the columns of a matrix product are linear combination of the columns of the left-hand factor, you can use this information to generate zeros in the last two columns of the product: $$\begin{bmatrix}-2&0&0&0&-2\\0&1&-2&0&0\\-2&0&0&1&1\\0&1&-2&1&3\end{bmatrix}\begin{bmatrix}1&0&0&0&1\\0&1&0&-2&0\\0&0&0&-1&0\\0&0&1&0&3\\0&0&0&0&-1\end{bmatrix} = \begin{bmatrix}-2&0&0&0&0\\0&1&0&0&0\\-2&0&1&0&0\\0&1&1&0&0\end{bmatrix}.$$ From here you can proceed as above to construct the left-hand matrix, and again, you’ll need to invert both to get $S$ and $T$. (In this case, inverting the right-hand matrix is quite simple: it will be a permutation of the matrix that you’ve built.)
