I want to compute the following integral involving complex exponentials but my approach is leading me to problems with infinities.
$$ I = {a \over 2} \int_{-\infty}^\infty (e^{-bt^2+i \omega t} + e^{-bt^2-i \omega t})\; dt $$
$$ = {a \over 2} \bigg({e^{-bt^2+i \omega t} \over -2bt+i \omega}\bigg|_{-\infty}^\infty + {e^{-bt^2-i \omega t} \over {-2bt-i \omega}}\bigg|_{-\infty}^\infty\bigg) $$
This leads to problems with infinities.
In general, it is easier to see here that the integrals are Fourier transforms.
$${a \over 2} \int_{-\infty}^\infty (e^{-bt^2+i \omega t} + e^{-bt^2-i \omega t})\; dt =\\ = \frac{a}{2} \sqrt{2 \pi} \left(\frac{1}{\sqrt{2 \pi}}\int_{-\infty}^\infty e^{-bt^2+i \omega t} dt + \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^\infty e^{-bt^2-i \omega t} dt \right) =\\ =\frac{a}{2} \sqrt{2 \pi} \left( \mathcal{F}(e^{-bt^2})(\omega) + \mathcal{F}^{-1}(e^{-bt^2})(\omega) \right) = \frac{a}{2} \sqrt{2 \pi} \frac{\sqrt{2} e^{-\frac{\omega ^2}{4 b}}}{\sqrt{b}} = \frac{\sqrt{\pi } a e^{-\frac{\omega ^2}{4 b}}}{\sqrt{b}},$$ where $\mathcal{F}$ is a Fourier transform and $\mathcal{F}^{-1}$ is a inverse Fourier transform. We can find Fourier transforms for the desired functions from the special tables. Tables of some Fourier transforms can be viewed, for example, on Wikipedia.
Let's consider the general case for the Gaussian integral: $$\int_{-\infty}^{\infty}e^{ax^2 + bx + c} dx$$
where $a,b,$ and $c$ are complex and $\text{Re}(a) < 0.$
We can begin by completing the square in the exponent to bring the integral closer to the original $\int e^{-x^2} dx$ case: just as in the real numbers, we have $ax^2 + bx + c = a(x + \frac{b}{2a})^2 +(c - \frac{b^2}{4a}).$ Plugging this in:
$$\int_{-\infty}^{\infty}e^{ax^2 + bx + c} dx = \int_{-\infty}^{\infty}e^{a(x + \frac{b}{2a})^2 +(c - \frac{b^2}{4a})} dx = e^{c - \frac{b^2}{4a}} \cdot \int_{-\infty}^{\infty}e^{a(x + \frac{b}{2a})^2} dx$$
using the exponent rule $e^{x + y} = e^x + e^y$ and pulling out a constant by the linearity of the integral.
Now we can proceed with the typical argument: $$I = \int_{-\infty}^{\infty}e^{a(x + \frac{b}{2a})^2} dx, I^2 = \int_{-\infty}^{\infty}\int_{-\infty}^{\infty} e^{a(x + \frac{b}{2a})^2 + a(y + \frac{b}{2a})^2} dy dx = \iint_{\mathbb R^2} e^{a(x + \frac{b}{2a})^2 + a(y + \frac{b}{2a})^2} dA$$
We can modify our switch to polar coordinates slightly to accommodate for the extra term: let $x = r\cos\theta - \frac{b}{2a}$ and $y = r\sin\theta - \frac{b}{2a}.$ Because the only change is a constant, the derivatives of $x$ and $y$ are unchanged, and so is the Jacobian:
$$J = \begin{bmatrix}x_r & x_\theta \\ y_r & y_\theta\end{bmatrix} = \begin{bmatrix}\cos\theta & -r\sin\theta \\ \sin\theta & r\cos\theta\end{bmatrix}$$ $$\det J = (\cos\theta)(r\cos\theta) - (\sin\theta)(-r\sin\theta) = r \cos^2\theta + r \sin^2\theta = r$$ $$I^2 = \int_0^{2\pi}\int_0^\infty e^{ar^2} r dr d\theta$$
We can recognize the innermost integrand as $\frac1{2a} \frac{d}{dr} e^{ar^2},$ so using the fundamental theorem of calculus and the defintion of our indefinite integral we have $\int_0^\infty e^{ar^2} r dr = \lim_{b \to \infty} \frac{1}{2a} (e^{ab^2} - 1).$ By linearity (assuming for the moment that the required limits exist) this is equivalent to $\frac1{2a}[(\lim_{b \to \infty} e^{ab^2}) - 1].$
Let $a = s + it$ for real $s,t,$ noting that we've required $s < 0.$ Applying Euler's theorem, we have that $e^{ab^2} = e^{sb^2}\cdot e^{itb^2} = e^{sb^2}(\cos(tb^2) + i\sin(tb^2)).$ The squeeze theorem can now be applied to the real and imaginary parts to show that both go to $0,$ so the limit is $0.$
So we have:
$$I^2 = \int_0^{2\pi} -\frac{1}{2a} d\theta = -\frac\pi a \to I = \sqrt{- \frac \pi a}$$
and plugging back into our original form:
$$\int_{-\infty}^{\infty}e^{ax^2 + bx + c} dx = e^{c - \frac{b^2}{4a}}\sqrt{\frac \pi{-a}}$$
for $\text{Re}(a) < 0.$
Applying this to your case in particular we get $$\int_{\mathbb R} e^{-at^2}\cos(\omega t) dt = \int_{\mathbb R}\frac12(e^{-at^2+i\omega t} + e^{-at^2-i\omega t}) dt = \frac12\left(e^{-\frac{(i\omega)^2}{4(-a)}}\sqrt{\frac\pi {-(-a)}} + e^{-\frac{(-i\omega)^2}{4(-a)}}\sqrt{\frac\pi {-(-a)}}\right) = \boxed{e^{-\frac{\omega^2}{4a}}\sqrt{\frac\pi a}}$$
We can corroborate this result using an alternative method. Let $I(a, \omega) = \int_{-\infty}^{\infty} e^{-at^2}\cos(\omega t) dt.$ By the above style of argument we know that $I(a, 0) = \sqrt{\frac \pi a}.$
Now suppose we were to take a partial derivative with respect to $\omega.$ By the Leibniz rule of integration, we should be able to pull the derivative into the integral as follows:
$$\frac{\partial}{\partial\omega} I = \int_{-\infty}^{\infty} \frac{\partial}{\partial\omega} e^{-at^2}\cos(\omega t) dt = \int_{-\infty}^{\infty} -te^{-at^2}\sin(\omega t) dt$$
We can use the extra $t$ to do integration by parts:
$$u = \sin(\omega t), dv = -te^{-at^2} dt$$ $$du = \omega \cos(\omega t) dt, v = \frac1{2a}e^{-at^2}$$
$$\int_{-\infty}^{\infty} -te^{-at^2}\sin(\omega t) dt = \frac1{2a}\sin(\omega t)e^{-at^2}\Big|^{\infty}_{-\infty} - \int_{-\infty}^{\infty} \frac\omega{2a}e^{-at^2}\cos(\omega t) dt$$
Note that the integral we end up with is the same as our definition of $I,$ so substituting it in as $I$ we get:
$$\frac{\partial}{\partial \omega} I = -\frac{\omega}{2a} I$$
which can easily be solved as follows:
$$\frac1I \frac{\partial}{\partial \omega} I = -\frac{\omega}{2a}$$ $$\frac{\partial}{\partial \omega} (\ln I) = -\frac{\omega}{2a}$$ $$\ln I = -\frac{\omega^2}{4a} + f_1(a)$$ $$I = f_2(a)e^{-\frac{\omega^2}{4a}}$$ $$I(a, 0) = \sqrt{\frac \pi a} \Rightarrow I(a, \omega) = \sqrt{\frac \pi a} \cdot e^{-\frac{\omega^2}{4a}}$$
