The AM-GM inequality
$$
\sqrt{ab}\le\frac{a+b}{2}
$$
becomes less and less tight as the numbers $a$ and $b$ become further and further apart. Indeed, we only get equality if $a=b$.
In your example, we would get equality in the inequality
$$
9\sec^2x + \cos^2x \ge 6
$$
if $9\sec^2x$ were equal to $\cos^2x$ for some $x$. But that never happens - $9\sec^2x$ is always at least $9$ and $\cos^2x$ is always at most $1$.
What this means is that the theoretical lower bound of $6$ -
$$
9\sec^2x + \cos^2x \ge 6
$$
- is never attained. Indeed, since $9\sec^2x$ and $\cos^2x$ are so far apart, the left hand side is actually always substantially larger than $6$. As you point out, it is actually at least $9$.
What's the moral? Use the AM-GM inequality if all you want to know is that one thing is greater than or equal to another. If you're interested in a minimum value that is actually attained, then you'd better make sure that the values you use in the inequality are close together. Otherwise, you'd do better to use another method.
In this case, you're better off using specific analytic properties of the $\cos$ and $\sec$ functions. For example, you might differentiate the expression $\cos^2x+9\sec^2x$. Or you could try to plot the curve and see where it attains its minimum value.