So I tried solving this: https://brilliant.org/practice/level-2-4-limits-of-functions/?p=1
Got this:
I'm completaly curious to know if this answer marked in green is right (which is the right answer according to the website). For me, the limit shouldn't exist, but if that wasn't the answer, it could, at best, be 6. Intuitively, saying that the answer is 5 would imply that you're taking the limit x -> 2- over x -> 2+ which doesn't make sense to me.
This one is tricky, but it appears the given answer is indeed right, if you follow the definition.
Suppose you plug in to x numbers arbitrarily close to 6, but smaller than 6. It's clear that f(x) is slightly greater than 2, and thus f(f(x)) will be slightly less than 5.
However, do the same thing with numbers arbitrarily close to 6, but larger than 6. The same result holds -- f(x) will be slightly greater than 2, and thus f(f(x)) is still less than 5.
Since the limit from the left and right are equal, the limit exists in general.
You not only have $f(x) \to 2$ for $x \to 6$ but even $f(x) \downarrow 2$ for $x \to 6$. Because $\lim_{y \downarrow 2} f(y) = 5$ you get that $$ \lim_{x \to 6} f(f(x)) = \lim_{y \downarrow 2} f(y) = 5. $$
Those answers are correct.
The key is that $f(x) > 2$ for all $x$ close to but not equal to $6$.
This means that $f(f(x))$ always approaches $f(2)$ from above, so you look only at the right hand limit.
If the curve crossed $y=2$ so $f(x)$ for $x > 6$ was less than $2$, the limit would not exist.
Informally, $\lim_{x \to p} f(x)=L$ means "when f is applied to any input sufficiently close to p, the output value is forced arbitrarily close to L". In the picture, when $x$ is sufficiently close to 6, $f(x)$ can be arbitrarily close to $2$ and always larger than $2$. Then $f(f(x))$ can be arbitrarily close to $5$, since the curve in $x\leq2$ is not used at all.
$\lim_{x \to 6} f(f(x))=L$ is defined to mean:
For all $\varepsilon>0$, there exists a $\delta>0$ such that for all $x$ (in $D$) that satisfy $0<|x-6|<\delta$, the inequality $|f(f(x))-L|<\varepsilon$ holds.
Let $L=5$. For all $\varepsilon>0$, we select $\delta$ using the following steps:
Now, for all $x$ that satisfy $0<|x-6|<\delta$, $x$ is nearer to $6$ than both $c_1$ and $c_2$, so $c_1<\delta<c_2$ (and $\delta \neq 6$). Therefore from the picture, $2<f(x)<d$, and $5>f(f(x))>5-e$, so $|f(f(x))-5|<e\leq\varepsilon$ holds.
Thus from the definition $\lim_{x \to 6} f(f(x))=5$.
Take a look at two different examples using similar functions and the same idea as in the original problem: in first the example, the limit DNE; in the second, however, the limit exists and is equal to 7. Click here to see the examples
