($8$ points) Suppose that $$f(x) = \frac1{x - 3} \quad \text{and} \quad g(x) = \frac{x - 7}{x + 5}$$ For each function $h$ given below, find a formula for $h(x)$ and the domain of $h$. Use interval notation.
(A)$\hspace{0.2cm}$ $h(x) = (f \circ g)(x)$
I suppose that the domain would be $(−\infty,−11) \cup (−11, \infty)$. However it shows me wrong. Could anyone help to obtain the correct answer?
The domain of $f\circ g$ is $\mathbb{R}\setminus\{-11,-5\}=(-\infty,-11)\cup(-11,-5)\cup(-5,+\infty)$. That's because:
Let $h(x)=f(g(x))$.
Thus, by definition $$D(h)=D(g)\cap\{x|g(x)\in D(f)\}.$$
$$h(x)=f(g(x))=\frac{1}{\frac{x-7}{x+5}-3}=-\frac{x+5}{2x+22}$$ and the domain of $h$ is $\mathbb R\setminus\{-11,-5\}$.
$(f\cdot g)(x)$ can be rewritten as $f(g(x))$, which implies that you are computing the function $f$ of $g(x)$, and since $g(x) = \displaystyle \frac{x-7}{x+5}$, it can be inferred that you are in essence computing $f\left(\displaystyle \frac{x-7}{x+5}\right)$. After you've computed this, you should find the values where $g(f(x))$ is not defined, and then exclude them from your domain. Hopefully this gives you some insight?
I can see why you came to the answer $(-\infty,-11) \cup (-11, \infty)$. That's because if you compute your $f\circ g$ as a symbolic expression, you get: \[\frac{1}{\frac{x-7}{x+5}-3} = \frac{1}{\frac{-2x-22}{x+5}} = \frac{x+5}{-2(x+11)}.\] On this face of it this is perfectly well-defined at $-5$.
However, if you decide that $g$ is not defined at $-5$, you might then conclude that $f\circ g$ (the function "apply $g$ and then $f$") can't be defined at $-5$, since after all you can't apply $g$ in that case.
A relatively sophisticated perspective on this is that "$f \circ g$ has a removable singularity at $-5$", that is to say, a point at which the function is technically not defined, but for which there is a perfectly good value (and indeed, only one value) the function could have that would be consistent with its behaviour elsewhere – in this case, you can extend $h$ by setting $h(-5) = 0$.
In fact, we tend to come across this kind of situation all the time, and usually we just plain ignore it – e.g. when cancelling $z$ from $xz/yz$, we rarely bother to say "except when $z = 0$". It's rarely problematic to just remove all removable singularities as soon as they turn up: having a function defined on a bigger set than you strictly need isn't a problem because you can always just restrict it to a smaller one.
However, in a question like this which is specifically asking about domains, it pays to be precise – and that means that whenever you replace a function with its definition, you must ensure that the argument belongs to its domain, possibly restricting your own domain as necessary.
In order to "evaluate" $f(g(x))$, We need all $x$ for which $g(x)$ exists AND for which $f(g(x))$ exists.
$g(x)$ exists unless $x=-5$.
$f(g(x))$ exists unless
\begin{align} g(x) &= 3 \\ \dfrac{x-7}{x+5} &= 3 \\ x &= -11 \end{align}
So the domain is $\mathbb R \setminus \{-5,-11\}$
g(x)is undefined when x=-5.Now,h(x) is undefined when x=-11. Since g(x) is an input function, it cannot take value of -11.Solve g(x)=(x-7)/(x+5)=-11. Solution: x = 6. So, final answer: domain of h(x) is all real numbers except x= 5 and x=-6.
