It suffices to prove that if $h(x)=g(x)-f(x)$ satisfies $\int_a^b hdx\geq 0$, then there exists $c\in[a,b]$ such that $h(c)\geq 0$ by linearity of the integral. By contradiction, assume $g(x)<0$ for all $x\in [a,b]$. Then by the definition, for any tagged partition $\mathcal{P}=\{a=x_0<x_1<\dots<x_n=b, t_i\in[x_{i-1},x_i]\:i\in[1,n]\}$, we have $$R(\mathcal{P})=\sum_{i=1}^n h(t_i)(x_{i}-x_{{i-1}})<0\;\;\;\;\;(1)$$Since $x_i-x_{i-1}>0$ and $h(t_i)<0$, so each term of the sum is less than $0$, and thus there sum is also less than $0$.
Now we consider two scenarios: $\int_a^b hdx>0$ and $\int_a^bhdx=0$. Let $s=\int_a^bhdx$. By definition, the integral is defined if, $\forall \epsilon>0$, there is $\delta>0$ and a tagged partition $\mathcal{P}$ such that $\max(x_{i}-x_{i-1})<\delta$ and $$|R(\mathcal{P})-s|<\epsilon\;\;\;\;\;\;(2)$$ In the first case, $s>0$, choose $\epsilon=s/2$. Then the above condition implies $$R(\mathcal{P})>-\epsilon+s=s/2>0$$which contradicts (1). Now suppose $s=0$. It suffices to prove the negation of (2), which is basically the statement that $R(\mathcal{P})$ cannot get too close to $0$. Since $h$ is Riemann integrable, it is bounded, so $\max(h)<M$ for some $M< 0$. Let $\mathcal{P}$ be any tagged partition. Then $$R(\mathcal{P})=\sum_{i=1}^n h(t_i)(x_i-x_{i-1})\leq M\sum_{i=1}^n(x_i-x_{i-1})= M(b-a)$$because the sum $\sum_{i=1}^n (x_i-x_{i-1})$ is telescoping. So let $\epsilon=-M(b-a)/2$. Then we can find no partition such that $R(\mathcal{P})\in (-\epsilon,\epsilon)$ by the argument I just made (work out the details if you don't see it). So this contradicts integrability, and thus there is $c\in [a,b]$ such that $h(c)\geq 0$.
Edit: As noted in the comments (and a detail I attempted to skirt past), that $h(x)<0$ for all $x\in [a,b]$ does not imply $\sup_{[a,b]}h=M<0$. It could be that $M=0$, in which case my argument doesn't work. In that scenario, using an argument that the function is continuous at some point is probably best. I was trying to do this primarily from the definition.
Edit 2: So it is possible to do this without appealing to continuity, according to this answer. However, their definition of the Riemann integral is much easier to use for this purpose, but it must be possible here as well.