What is the difference actually between big O and small o in numerical methods?

Question

I am studying numerical methods for ODE. I am confused about the "o"s there. What is the difference actually between big O and small o in numerical methods? I have read somewhere that the big O means the convergence order. Am I right? If so, what about small o then?

Answer 1

The big $O$ notation is usually used in evaluating the complexities of algorithms. A function $f$ is said to be $O(g)$ for a function $g$ if there exist constants $k$ and $C$ such that $f(n) < kg(n) + C$ for all large enough $n$. For example, if you make an algorithm that takes $2n^2+172n + 1$ steps to complete for an input of size $n$, you say that it takes $O(n^2)$ steps, as the $n^2$ factor dominates the polynomial for large $n$.

The small $o$ means something else. It is used to evaluate how quickly a function drops towards $0$ when $x$ approaches some number. A function $f$ is said to be $o(g)$ around $a$ if the limit $$\lim_{x\rightarrow a}\frac{f(x)}{g(x)}$$ exists. For example, the function $\sin x$ is $o(x)$, but not $o(x^2)$.