Assume that f is of class $C^1$ on $[a,b]$
My Work:
I am assuming that this problem largely has to do with integration by parts; however, I am unsure where to start applying IBP. I first attempted to apply IBP on $\int_a^b [f'(x)]^2$, but this led to no avail because it involved a second derivative, which is impossible due to the fact that the class is $C^1$.
Currently, I am working on applying IBP to$\int_a^b [xf(x)]^2dx$, but I am not making much progress.
Bu Cauchy Scwhartz, $(*) \vert \int_a^b xf(x)f'(x) dx \vert^2\leq \int_a^b (x(f(x))^2dx.\int_a^b(f'(x)^2) dx$. The LHS of $(*)$ can be computed (IBP)as $ \int_a^b xf(x)f'(x) dx = x{f^2 \over 2}\vert_a^b-{1\over 2}\int_a^bf^2(x)dx=-1/2$. Hence the LHS of $(*)$ is exactly $1/4$. Fo strict inequality note that we have $=$ in CS iff the two terms are proportional i.e. $x.f(x)=\alpha f'(x)$. A solution of this linear ODE cannot vanish at $a$ and $b$ simultaneously.
By partial integration $$ 1=\int_a^b f^2(x)\:dx=xf^2(x)\bigg|_a^b-2\int_a^b xf(x)f'(x)\:dx=-2\int_a^b xf(x)f'(x)\:dx $$ By Cauchy--Schwarz inequality $$ \left|\int_a^b xf(x)f'(x)\:dx\right|^2\leqslant\int_a^b x^2f^2(x)\:dx\int_a^b f'^2(x)\:dx $$ Thus $$ \int_a^b x^2f^2(x)\:dx\int_a^b f'^2(x)\:dx>\frac1{4} $$ Note that equality can not hold since that would mean $f'(x)=kxf(x)$. By integration we get $$ f(x)=Ce^{\frac{kx^2}{2}} $$ Since $f(a)=0$, $C=0$. So $f(x)=0$, but this contradicting $\int_a^b f^2(x)\:dx=1$.