Since you know some ratios of lengths, it will be nicer to use barycentric coordinates instead of cartesian coordinates. So let the normalized barycentric coordinates of $A', B'$ and $C'$ wrt to $\bigtriangleup ABC$ be $(\alpha_1, \beta_1, \gamma_1), (\alpha_2, \beta_2, \gamma_2)$ and $(\alpha_3, \beta_3, \gamma_3)$. Since $\frac{CM}{MA}=\mu$ and $\frac{AN}{NB}=\nu$ we get from $A'=BM\cap CN$
$$\frac{\alpha_1}{\gamma_1}=\mu, \hspace{0.5cm} \frac{\beta_1}{\alpha_1}=\nu$$
Together with $\alpha_1+\beta_1+\gamma_1=1$, this yields
$$(\alpha_1, \beta_1, \gamma_1)=\left(\frac{\mu}{1+\mu+\mu\nu}, \frac{\mu\nu}{1+\mu+\mu\nu}, \frac{1}{1+\mu+\mu\nu}\right)$$
Similarly,
$$(\alpha_2, \beta_2, \gamma_2)=\left(\frac{1}{1+\nu+\lambda\nu}, \frac{\nu}{1+\nu+\lambda\nu}, \frac{\lambda\nu}{1+\nu+\lambda\nu}\right)$$
$$(\alpha_3, \beta_3, \gamma_3)=\left(\frac{\lambda\mu}{1+\lambda+\lambda\mu}, \frac{1}{1+\lambda+\lambda\mu}, \frac{\lambda}{1+\lambda+\lambda\mu}\right)$$
Now, using your notation, we get
$$\begin{split}
&\frac{S'}{S}(1+\lambda+\lambda\mu)(1+\mu+\mu\nu)(1+\nu+\nu\lambda) \\=\,\,
&\begin{vmatrix}
\alpha_1 & \beta_1 & \gamma_1 \\
\alpha_2 & \beta_2 & \gamma_2 \\
\alpha_3 & \beta_3 & \gamma_3
\end{vmatrix}(1+\lambda+\lambda\mu)(1+\mu+\mu\nu)(1+\nu+\nu\lambda) \\=\,\,
&\begin{vmatrix}
\alpha_1 & \beta_1 & 1 \\
\alpha_2 & \beta_2 & 1 \\
\alpha_3 & \beta_3 & 1
\end{vmatrix}(1+\lambda+\lambda\mu)(1+\mu+\mu\nu)(1+\nu+\nu\lambda) \\=\,\,
&(\alpha_1\beta_2+\alpha_2\beta_3+\alpha_3\beta_1-\alpha_1\beta_3-\alpha_2\beta_1-\alpha_3\beta_2) \\&\cdot(1+\lambda+\lambda\mu)(1+\mu+\mu\nu)(1+\nu+\nu\lambda) \\=\,\,
&\mu\nu(1+\lambda+\lambda\mu)+(1+\mu+\mu\nu)+\lambda\mu^2\nu(1+\nu+\lambda\nu) \\ &-\mu(1+\nu+\lambda\nu)-\mu\nu(1+\lambda+\lambda\mu)-\mu\lambda\nu(1+\mu+\mu\nu) \\=\,\,
&\mu\nu+\lambda\mu\nu+\lambda\mu^2\nu+1+\mu+\mu\nu+\lambda\mu^2\nu+\lambda\mu^2\nu^2+\lambda^2\mu^2\nu^2 \\ &-\mu-\mu\nu-\lambda\mu\nu-\mu\nu-\lambda\mu\nu-\lambda\mu^2\nu-\lambda\mu\nu-\lambda\mu^2\nu-\lambda\mu^2\nu^2\\=\,\,
&\lambda^2\mu^2\nu^2-2\lambda\mu\nu+1=(\lambda\mu\nu-1)^2,
\end{split}$$
which is what we wanted.