The formulas for $\alpha_n,\beta_n$ need a factor of $r^n$ on the left. When corrected, they will follow from
$$a_n r^n = \frac{1}{2\pi}\int_0^{2\pi} f(re^{i\theta}) e^{-in\theta}\,d\theta$$
by separation of the real and imaginary parts.
To estimate integrals of the form $\int_0^{2\pi}u(r,\theta)\cos(n\theta)d\theta$ in terms of $\max u$, it would help to have a nonnegative factor next to $u$. But we have $\cos(n\theta)$. there. To fix this, add $1$ to the cosine.
$$
\int_0^{2\pi}u(r,\theta)\cos(n\theta)\,d\theta
= \int_0^{2\pi}u(r,\theta)(1+\cos(n\theta))\,d\theta - \int_0^{2\pi}u(r,\theta)\,d\theta $$
The first integral is estimated using $(1+\cos(n\theta))\le 2$:
$$\int_0^{2\pi}u(r,\theta)(1+\cos(n\theta))\,d\theta \le 4\pi \max(A(r),0)$$
(can't use negative $A(r)$, because $u(r,\theta)(1+\cos(n\theta)) \le 2A(r)$ would not be a valid inequality).
The second integral is evaluated using the mean value property:
$$\int_0^{2\pi}u(r,\theta)\,d\theta = 2\pi \operatorname{Re}f(0)$$
In conclusion, $$\alpha_n\le 4\max(A(r),0)-2\operatorname{Re}f(0) \tag{1}$$
To upgrade (1) to an estimate for $|a_n|$, use the "rotate until it's positive" trick. Namely, apply (1) to $f(\omega z)$ where $\omega $ is a unimodular complex number chosen so that for the rotated function, $a_n$ is a positive number.