Apart from the particular goal, it is important to note that a version of strong convexity can be defined without referring to derivatives. Let's say that $f$ is generalized strongly convex (GSC) if there is $\epsilon>0$ such that $f(x) - \epsilon x^2$ is convex.
It is easy to see that a GSC function is bounded from below. Indeed, as $x\to \pm\infty$, the difference $f(x) - \epsilon x^2$ can go to $-\infty$ at most linearly, and therefore $f(x)\to+\infty$.
It remains to show that strongly convexity a.e. implies GSC. Consider the Lebesgue-Stiltjes measure $d(f'(x))$. By convexity of $f$, this measure is positive. Hence it decomposes into absolutely continuous part (which has density $\ge \epsilon$ by assumption) and nonnegative singular part. It follows that
$$ f'(b)-f'(a)\ge \epsilon( b-a) ,\qquad a<b $$
In other words, $f'(x)-\epsilon x$ is nondecreasing. Thus, $f(x)-\frac12 x^2$ is convex.
Note that I interpreted your definition of strong convexity a.e. as including convexity of $f$. If we only have $f''\ge \epsilon $ a.e., then nothing much can be said about the global behavior of $f$: one could subtract some rapidly growing version of the Cantor function from $f$.