Here is a solution that makes the assumption $X_p = 0$ very transparent.
The geodesic vector field has a flow $G\colon I\times TU\to TM$ (cf. do Carmo p. 65), where by definition,
$$
\gamma(1,p,v) \equiv (\pi\circ G)(1,p,v) = \exp_pv,
$$
where $\pi\colon TM\to M$ is the canonical projection. It will be convenient in what follows to use this $\gamma$. If $\theta\colon (-\varepsilon,\varepsilon)\times U\to M$ is the flow of $X$, we will also make use of the two maps
\begin{align*}
q&\mapsto \theta_t(q) \equiv \theta(t,q),\\
t&\mapsto \theta^{(q)}(t) \equiv \theta(t,q).
\end{align*}
For each fixed $q\in U$, $\big(\theta^{(q)}\big)'(0) = X_q$.
Let $S_\delta(p)\subset U$ be a geodesic sphere centered at $p$, and let $q\in S_\delta(p)$. Then $q$ is the endpoint $(s=\delta)$ of a certain unit-speed geodesic—concretely there is some $v\in S_1(0)\subset T_pM$ so that $q = \gamma(\delta,p,v)$. By the assumption that $X$ is Killing, for every fixed $t\in(-\varepsilon,\varepsilon)$, $\theta_t$ is an isometry, so
$$
[0,\delta]\ni s\longmapsto \gamma\big(s,p,(d\theta_t)_p(v)\big)
$$
is a unit-speed geodesic.
Consider the parametrized surface (cf. Definition 3.3 of do Carmo, p. 67)
$$
f(s,t) \equiv (\theta_t\circ\gamma)(s,p,v).
$$
For each $t$, the curve $s\mapsto f(s,t)$ is a unit-speed geodesic starting from $p$ because $\theta_t$ is an isometry. By the chain rule,
$$
\frac{\partial f}{\partial s}(0,t) = (d\theta_t)_{p}\bigg(\frac{\partial}{\partial s}\gamma(0,p,v)\bigg) = (d\theta_t)_p(v).
$$
By definition of $\gamma$, and our observation that $s\mapsto f(s,t)$ is a unit-speed geodesic with endpoint ($s=\delta$) in $S_\delta(p)$, we thus see that
$$
f(s,t) = \gamma(s,p,(d\theta_t)_p(v)).
$$
For each fixed $t$, the curve $s\mapsto f(s,t)$ is a geodesic by the discussion above, so we have for all $(s,t)$,
$$
\frac{D}{\partial s}\frac{\partial f}{\partial s} = 0.
$$
Moreover, $\frac{\partial f}{\partial s}(\delta,0) = \frac{\partial}{\partial r}\big|_q$.
Looking at the definition, we see that $\frac{\partial f}{\partial t}(\delta,0) = X_q$. Now we see that we would like to show that
$$
\left< \frac{\partial f}{\partial s}(\delta,0),\frac{\partial f}{\partial t}(\delta,0)\right> = 0.
$$
For all $(s,t)$, by the lemma of symmetry (cf. do Carmo p. 68)
\begin{align*}
\frac{\partial}{\partial s}\left< \frac{\partial f}{\partial s},\frac{\partial f}{\partial t}\right> &= \left< \underbrace{\frac{D}{\partial s}\frac{\partial f}{\partial s}}_{=\ 0},\frac{\partial f}{\partial t}\right> + \left< \frac{\partial f}{\partial s},\frac{D}{\partial s}\frac{\partial f}{\partial t}\right>\\
&= \left< \frac{\partial f}{\partial s}, \frac{D}{\partial t}\frac{\partial f}{\partial s}\right> & \text{(symmetry)}\\
&= \frac{1}{2}\frac{\partial}{\partial t}\left< \frac{\partial f}{\partial s},\frac{\partial f}{\partial s}\right>.
\end{align*}
Now, for any fixed $s$, $\left< \frac{\partial f}{\partial s},\frac{\partial f}{\partial s}\right>$ is constant as a function of $t$ because $\theta_t$ is an isometry, so the last expression in the above display is identically $0$.
We thus deduce in particular that $\left< \frac{\partial f}{\partial s}(s,0),\frac{\partial f}{\partial t}(s,0)\right>$ is constant as a function of $s\in[0,\delta]$. By smooth dependence on initial conditions,
\begin{align*}
\lim_{s\to 0} \frac{\partial f}{\partial t}(s,0) = \lim_{s\to 0}\big(\theta^{(\gamma(s,p,v))}\big)'(0) = \big(\theta^{(p)}\big)'(0) = X_p = 0,
\end{align*}
so we conclude that $\left< \frac{\partial f}{\partial s}(\delta,0),\frac{\partial f}{\partial t}(\delta,0)\right> = 0$, which was our goal.
We essentially reproduced a large portion of the proof of Gauss' lemma (cf. do Carmo p. 69), so there may be a way to shorten this proof using Gauss' lemma.