I believe that you are trying to solve a nonlinear equation of the form $$A(x)x = b,$$
where $A : \mathbb{R}^n \rightarrow \mathbb{R}^n \times \mathbb{R}^n$ and $b \in \mathbb{R}^n$ with respect to the unknown vector $x \in \mathbb{R}^n$.
The iteration that you describe takes the form
$$ x_{k+1} = A(x_k)^{-1} b.$$
This is an example of a functional iteration, i.e., an iteration of the form $$ x_{k+1} = g(x_k).$$
In your case $g : \Omega \rightarrow \mathbb{R}^{n \times n}$ is given by $$g(x) = A(x)^{-1}b$$ where $x \in \Omega$ and $\Omega \subseteq \mathbb{R}^n$ is some suitable range of domain.
Functional iterations are also known as fixed point iterations.
Let $\Omega \subseteq \mathbb{R}^n$ be closed. If $g : \Omega \rightarrow \Omega$ is a contraction, i.e., if there exists $L \in [0,1)$ such that $$\forall x, y \in \mathbb{R}^n \: : \: \|g(x) - g(y)\| \leq L \|x-y\|$$ then $g$ has a unique fixed point $r \in \Omega$ and the functional iteration will converge to $r$ regardless of the choice of $x_0 \in \Omega$. This is a special case of the contraction mapping theorem which is also known as Banach's fixed point theorem.
In your case, you must investigate if there exists $L \in [0,1)$ such that $$\|A(x)^{-1} b - A(y)^{-1}b\| \leq L\|x-y\|.$$
The specific form of your function $x \rightarrow A(x)$ is needed to complete the analysis. However, this is a subject suitable for a new question.