Condition number interpretation

Question

I have a (nonlinear) problem with two variables, for which I computed a relative condition number as $$K_{rel}(x_1,x_2) = \max\{1, c\},$$ where I had $$\Bigg| \frac{f(x_1, x_2) - f(\tilde{x_1},\tilde{x_2})}{f(x_1, x_2)} \Bigg| = \Bigg| 1 \cdot \frac{x_1-\tilde{x_1}}{x_1} + c \cdot \frac{x_2-\tilde{x_2}}{x_2}\Bigg|.$$

What is the interpretation of this? If $c<1$, is it valid to say that the problem is well-conditioned w.r.t. $x_2$ and ill-conditioned w.r.t. $x_1$? If I could keep $x_1$ fixed, would I get a well-conditioned problem? If the relative condition number equals 1, do I already have an ill-conditioned problem? (I only found out that it is well-conditioned for $<1$)

Answer 1

The value of condition number is that they measure the relative sensitivity of the output to small relative changes in the input.

Here the output is the function value $y = f(x_1, x_2)$ and the input is the vector $(x_1,x_2)$. If $c \gg 1$, then it would be proper to say that $f$ is ill-conditioned with respect to $x_2$, because the absolute value of relative error $\frac{y - \tilde{y}}{y}$ is roughly $c$ times the relative error on the input $\frac{x_2 - \tilde x_2}{x_2}$. In general, it would not be proper to say that $f$ is ill-conditioned with respect to $x_1$.

Obviously, a constant function has condition number $0$, so condition numbers less than $1$ are certainly possible, but in practice a condition number which is close to one is as good as it gets. Linear systems in real applications frequently have condition numbers larger than $10^{14}$ and push against the very limitations imposed by double precision arithmetic.

There is no precise definition of which condition number corresponds to a well-conditioned or an ill-conditioned problem, because the classification depends on what you need the result for as well as the available computer architecture.

Say, you are in charge of designing a skyscraper and the stiffness matrix has condition number $\kappa \sim 10^7$. If you only have a a single precision computer where the unit round off error is $u \sim 10^{-8}$, then you are looking at relative errors on the order of $\kappa u \sim 10^{-1}$, which should give you reason to be concerned, as there are lives riding on your calculation. I would call your problem ill-conditioned.

On the other hand, if you have a double precision computer available, then $u = 10^{-16}$ and you are looking at relative errors on the order of $\kappa u \sim 10^{-9}$. This number is ridiculous small compared with the size of the human errors that will be made during the actual construction. Therefore, I would consider the same computational problem well-conditioned.

Condition numbers are particularly useful because they establishes theoretical limitations on the accuracy of any computer implementation.