Let $f(t) = t + \phi(t)$. The condition on the integral of $f^2$ becomes $\forall x \in [1,2]$
$$ \int_1^x 2t\phi(t) + \phi^2(t) ~\mathrm{d}t \leq 0 $$
and in particular
$$ 0 \geq \int_1^x t \phi(t) ~\mathrm{d}t = \int_1^x \phi(t) \left(\int_1^t 1 \mathrm{d}y\right) ~\mathrm{d}t + \int_1^x \phi(t)~\mathrm{d}t $$
For convenience denote
$$ \Phi(x) = \int_1^x \phi(t) ~\mathrm{d}t $$
then the above expression can be rewritten as (by Fubini, noting that the domain of integration is $\{ (y,t) : 1 \leq y \leq t \leq x\}$)
$$ 0 \geq \int_1^x \int_y^x \phi(t) ~\mathrm{d}t ~\mathrm{d}y + \Phi(x) = \int_1^x \Phi(x) - \Phi(y) ~\mathrm{d}y + \Phi(x) $$
which we can simplify to
$$ \int_1^x \Phi(y) ~\mathrm{d}y \geq x \Phi(x) \tag{*}$$
Observe that by definition $\Phi(1) = 0$ and $\Phi$ is continuous as $\phi$ is Riemann integrable.
Proposition: $\Phi(x) \leq 0$ for every $x\in [1,2]$. Furthermore, if $\Phi(x_0) = 0$ then $\phi(x) = 0$ for every $x\in [1,x_0)$.
Proof:
To prove $\Phi(x) \leq 0$, assume for contradiction that there exists $x'$ such that $\Phi(x') > 0$. By continuity there exists $x'' \in (1,x']$ such that $\Phi$ is monotonic on $[1,x'']$ with $\Phi(x'') > 0$. This, however, would contradict (*), as we would have
$$ (x''-1) \Phi(x'') \int_1^{x''} \Phi(x'') \mathrm{d}y \geq \int_1^{x''} \Phi(y)\mathrm{d}y \geq x'' \Phi(x'') $$
which is absurd.
Now, suppose $\Phi(x_0) = 0$ for $x_0 > 1$. (*) implies
$$ \int_1^{x_0} \Phi(y) ~\mathrm{d}y \geq 0 $$
On the other hand from step 1 we know that $\Phi(y) \leq 0$ always. Hence this implies $\Phi(y) = 0$ for every $y\in [1,x_0)$. From the definition this implies $\phi(y) = 0$ for every $y\in [1,x_0)$.
Q.E.D.
Corollary: We have under the assumptions on the $L^2$ integral, that $\int_1^2 f(t) ~\mathrm{d}t \leq \frac32$ with equality only if $f(t) = t$.
Proof: $\int_1^2 f(t) \mathrm{d}t = \int_1^2 t \mathrm{d}t + \Phi(2) = \frac32 + \Phi(2)$ by definition. Use the above proposition. Q.E.D.
Edit:
The Fubini argument for integrating $(t-1)\phi(t)$ is as follows.
Observe that $t -1= \int_1^t 1 \mathrm{d}y$. So we can write the integral as a double integral
$$ \int_1^x (t-1) \phi(t) \mathrm{d}x = \int_1^x (\int_1^t 1 \mathrm{d}y ) \phi(t) \mathrm{d}t $$
Unfortunately, the domain of integration for the inner integral depends on the variable of integration of the outer integral. To deal with that, we re-write
$$ \int_1^t 1 \mathrm{d}y = \int_1^x \chi(t,y) \mathrm{d}y $$
where
$$ \chi(t,y) = \begin{cases} 1 & t > y \\
0 & t \leq y \end{cases} $$
In this form we can apply Fubini and interchange the order of integration
$$ \int_1^x (t-1) \phi(t) \mathrm{d}t = \int_1^x \int_1^x \chi(t,y) \phi(t) ~\mathrm{d}y ~ \mathrm{d}t = \int_1^x \int_1^x \phi(t) \chi(t,y) ~\mathrm{d}t ~ \mathrm{d}y $$
When we integrate in $t$ we have that $\chi(t,y) \neq 0$ only when $t > y$, so the inner integral can be replaced by
$$ \int_1^x \int_y^x \chi(t,y) \phi(t) ~\mathrm{d}t ~\mathrm{d}y $$
but on this domain we have that $\chi \equiv 1$ so we can drop it from the integral to get
$$ \int_1^x \int_y^x \phi(t) ~\mathrm{d}t ~\mathrm{d} y$$
and then applying the fundamental theorem of calculus to the inner integral we get, finally, as claimed
$$ \int_1^x \Phi(x) - \Phi(y) \mathrm{d}y $$
