Note that , by the continuity of $f$, for all $x$, there exists $c_x\in [a,x]$ such that $g(x)=f(c_x)$.
Now take $x_0\in ]a,b[$ and suppose that $g$ is not continuous at $x_0$. Then the limit of the increasing function $g$ is not $g(x_0)$ if $x\to x_0$, $x<x_0$, or $x\to x_0$, $x>x_0$. Suppose the first is true. As $g(x)$ is increasing, $g(x)$ has a limit $L<g(x_0)$ as $x\to x_0$, $x<x_0$. Hence we have $f(x)\leq g(x)\leq L<g(x_0)$ for all $x<x_0$. If $c_{x_0}<x_0$, we get a contradiction if we put $x=c_{x_0}$. Hence $c_{x_0}=x_0$. But then $f(x)\leq L<f(x_0)$ for all $x<x_0$ and this is a contradiction with the continuity of $f$ at $x_0$.
For the second case, we have $g(x)\geq L >g(x_0)$ for $x>x_0$. Then $x\geq c_x>x_0$ for $x>x_0$, and $f(c_x)\geq L>f(x_0)$, and we have a contradiction if $x\to x_0$, $x>x_0$.Same argument for $x_0=a$ or $b$.