There is this old unanswered question: https://math.stackexchange.com/questions/1025626/proof-the-fourier-transform-is-unitary-not-unitary
What is the easiest way to see that the Fourier transform is unitary and why it is important to have constant $1/\sqrt{2\pi}$ to make it unitary. Why $1/\sqrt{2\pi}$ does not work?
The second part of the question is whether there are other integral operators, i.e. $(Fg)(y) = \int g(x)k(x,y)dx$ which are unitary and where the function $k$ is real.
If $Q$ is unitary (i.e. $\langle Qx, Qy\rangle = \langle x, y\rangle$ for all $x,y$ in your space) and you multiply $Q$ by any constant $\lambda$, you get $$\langle \tilde Qx, \tilde Qy\rangle = \langle \lambda Q x, \lambda Q y\rangle = \lambda\bar\lambda\langle Qx, Qy\rangle = |\lambda|^2 \langle x,y\rangle$$ Thus the constants wich make $\tilde Q$ unitary iff $Q$ is unitary are those with magnitude $|\lambda| = 1$. For complex vector spaces these are characterized by $e^{i\theta}$. Thus if you accept the unitarity of $\mathcal F$ defined with the constant $\frac1{\sqrt{2\pi}}$, you already know all scalar multiples of $\mathcal F$ wich are unitary, namely the family $$\mathcal F_\theta[f] (\xi) := \frac{e^{i\theta}}{\sqrt{2\pi}} \int_{\mathbb R} f(x) e^{i\xi x}\ \mathrm dx \\ \mathcal F_\theta^{-1}[\hat f](x) = \frac{e^{-i\theta}}{\sqrt{2\pi}} \int_{\mathbb R} \hat f(\xi) e^{-i\xi x}\ \mathrm d\xi$$
Regarding your second question, many operators of the convolution type $$F[g](y) = (g\ast f)(y) = \int_{K} g(x)f(x \ominus y)\ \mathrm dx$$ with appropriate conditions on $f$ and an LCA group $K$ are examples.
