The key property is the formula for the derivative of products of functions holds also for the case of matrices.
$$ \frac{\partial}{\partial t} (C(t)D(t)) = \left(\frac{\partial}{\partial t} C(t)\right) \cdot D(t) + C(t) \cdot \frac{\partial}{\partial t} D(t). \tag{$\sharp$} $$
(But here we cannot change the order of the product, because we consider matrices. The proof is straightforward.)
If $D(t) = C(t)^{-1}$, then,
$$ O = \left(\frac{\partial}{\partial t} C(t)\right) \cdot C(t)^{-1} + C(t) \cdot \frac{\partial}{\partial t} C(t)^{-1}, $$
where $O$ denotes the zero matrix.
If additionally $C(0) = I$ (the identity matrix), then,
$$ O = \frac{\partial}{\partial t}|_{t=0} C(t) + \frac{\partial}{\partial t}|_{t=0} C(t)^{-1}. $$
The last equality you wrote follows from this.
By ($\sharp$),
$$ \frac{\partial}{\partial t}|_{t=0} (A(s)^{-1} B(t)^{-1} A(s) B(t)) = A(s)^{-1} \cdot \frac{\partial}{\partial t}|_{t=0} (B(t)^{-1} A(s) B(t)).$$
In the same manner, by using that $B(0) = B(0)^{-1} = I$,
$$\frac{\partial}{\partial t}|_{t=0} (B(t)^{-1} A(s) B(t)) = \left(\frac{\partial}{\partial t}|_{t=0} B(t)^{-1}\right) A(s) B(0) + B(0)^{-1} \cdot \frac{\partial}{\partial t}|_{t=0} (A(s) B(t))$$
$$ = \left(\frac{\partial}{\partial t}|_{t=0} B(t)^{-1}\right) A(s) + A(s) \cdot \left(\frac{\partial}{\partial t}|_{t=0} B(t)\right).$$
Thus we see that
$$ \frac{\partial}{\partial t}|_{t=0} (A(s)^{-1} B(t)^{-1} A(s) B(t)) = A(s)^{-1} \cdot \left(\frac{\partial}{\partial t}|_{t=0} B(t)^{-1}\right) A(s) + \left(\frac{\partial}{\partial t}|_{t=0} B(t)\right).$$
Hence,
$$ \frac{\partial}{\partial s}|_{s=0} \frac{\partial}{\partial t}|_{t=0} (A(s)^{-1} B(t)^{-1} A(s) B(t)) = \frac{\partial}{\partial s}|_{s=0} \left( A(s)^{-1} \cdot \left(\frac{\partial}{\partial t}|_{t=0} B(t)^{-1}\right) A(s) \right).$$
By using the assumption that $A(0) = A(0)^{-1} = I$, we see that
$$\frac{\partial}{\partial s}|_{s=0} \left( A(s)^{-1} \cdot \left(\frac{\partial}{\partial t}|_{t=0} B(t)^{-1}\right) A(s) \right) = \left(\frac{\partial}{\partial s}|_{s=0} A(s)^{-1}\right) \cdot \left(\frac{\partial}{\partial t}|_{t=0} B(t)^{-1}\right) + \left(\frac{\partial}{\partial t}|_{t=0} B(t)^{-1}\right) \cdot \left(\frac{\partial}{\partial s}|_{s=0} A(s)\right) $$
in the same manner. Thus we have the first equality.