OP is right
The functions are curves but the converse may not be true.
So for a curve like $x=g(y)$ one wouldn't ask for $g^{-1}(x)$ (it may not exist). Also, when $x=g(y)$ normally it is not easy to find $y_0$ for a given $x_0$, one has to solve the implicit equation.
Suppose the equation of the curve is $x=g(y) \implies dx=g'(y)dy$ and a point lying on that is knowm/given as $(x_0,y_0)$, then the equation of the thangent is
$$y-y_0=\frac{1}{g'(y_0)}(x-x_0).....(1)$$
Once the point is found, (1) represents the equation of tangent.
Example of the curve: For $x=y^y$ the equation of tangent at (4,2) is
$$ y-2=\frac{1}{4(1+\log 2)}(x-4)....(2)$$
because $g'(y)=y^y(1+\ln y)=x(1+\ln y)$
The point $(1,0)$ is a limit point, the Eq, of tangent on this is $y=0$. For $x=1$ there is another point (1,1). check the tangent at this point is $y=x.$
One can solve the implicit equation $y^y=2$ to get $y_0=1.55916...$ to write the tangent at $(2, 1.55961...)$ using (1).
See the curve $x=y^y$ and its tangents at points (4,2) and (1,1) in the fig. below
