First I do the inner integral:
$$
\int_0^{t_2} dt'' \, e^{-a(t_2-t'')} \, \delta(t''-t')
= \int_{-\infty}^{\infty} dt'' \, \chi_{[0, t_2]}(t'') \, e^{-a(t_2-t'')} \, \delta(t''-t')
= \chi_{[0, t_2]}(t') \, e^{-a(t_2-t')}
$$
Here, for any set $A$, $\chi_A(t) = 1$ if $t\in A$, $\chi_A(t)=0$ otherwise.
Then the outer integral:
$$
I(t_1,t_2)
= \int_0^{t_1} dt' \, e^{-a(t_1-t')} \, \chi_{[0, t_2]}(t') \, e^{-a(t_2-t')} \\
= e^{-a(t_1+t_2)} \int_0^{t_1} dt' \, e^{2at'} \, \chi_{[0, t_2]}(t') \\
= e^{-a(t_1+t_2)} \int_{-\infty}^{\infty} dt' \, \chi_{[0, t_1]}(t') \, e^{2at'} \, \chi_{[0, t_2]}(t') \\
= e^{-a(t_1+t_2)} \int_{-\infty}^{\infty} dt' \, \chi_{[0, \min(t_1,t_2)]}(t') \, e^{2at'} \\
= e^{-a(t_1+t_2)} \int_{0}^{\min(t_1,t_2)} dt' \, e^{2at'} \\
= e^{-a(t_1+t_2)} \left[\frac{1}{2a} e^{2at'} \right]_{0}^{\min(t_1,t_2)} \\
= \frac{1}{2a} e^{-a(t_1+t_2)} \left[e^{2a\min(t_1,t_2)} -1 \right]
$$