It depends on what is meant. Both are actually valid, in the following sense : if you're looking completely internally to $A$, then it makes more sense to say $\alpha (K)$.
However, if you have a more global view on all $K$-algebras, then you see that for any $K$-algebra $A$ and any polynomial $P\in K[X_1,...,X_n]$, there is an associated map $A^n\to A$, and you can say that $(a_1,...,a_n)$ generate $A$ if for any $a\in A$ there exists $P\in K[X_1,...,X_n]$ (so a polynomial with coefficients in $K$) such that the image of $(a_1,...,a_n)$ under the associated map is $a$.
The thing is that this associated map is in some sense "independent" of the algebra $A$, and it completely determines $P$ if you allow yourself to look at all $K$-algebras, so it makes sense to say that it is a polynomial with coefficients in $K$.
(PS : I'll add my personal opinion here, but I think that's not the best definition of finitely generated algebra; to me the subalgebra of $A$ generated by a subset $S$ should be the intersection of all subalgebras of $A$ containing $S$, so literally "the smallest subalgebra containing $S$", and then the description with polynomials should be a theorem/characterization, not the definition)