How to prove, that $$f(x)=\frac{||Ax-b||^2}{1-||x||^2}, ||x||^2<1, x \in \mathbb{R}^n$$ is convex?
UPDATE: I've tried the following approach. The function $$g(x) = log(||Ax-b||^2)$$ is convex (proof is straightforward), the same with function $$h(x) = log(\frac{1}{1-||x||^2}), ||x||^2<1$$ The affine combination of $g(x)$ and $h(x)$ is convex too. $$f'(x)=log(\frac{||Ax-b||^2}{1-||x||^2})$$ And applying increasing function $e^x$ preserves the convexity.