Yes. By definition, $\alpha$ is integral over $A$ if (and only if) there is a monic polynomial $p \in A[X]$ with $p(\alpha) = 0$.
Now, if $f_\alpha$ has coefficients in $A$, you have your monic polynomial in $A[X]$ of which $\alpha$ is a zero, hence $\alpha$ is integral over $A$.
Conversely, if $\alpha$ is integral over $A$, there is a monic polynomial $p \in A[X]$ with $p(\alpha) = 0$. By the definition of a minimal polynomial, that means that $f_\alpha \mid p$ in $K[X]$.
By one of the many lemmas of Gauss, if you have $p = f\cdot g$ where $p \in A[X]$ and $f,\,g \in K[X]$ are monic, you actually have $f,\, g \in A[X]$.