Let $M$ be a Riemannian manifold and $p$ be a point on $M$. Let $U$ be a normal neighborhood about $p$ (that is, the exponential map $\exp_p$ maps a neighborhood of the origin in $T_pM$ diffeomorphically onto $U$). Fix a vector $v_p\in T_pM$.
For each $q\in U$, let $v_q$ be the vector in $T_qM$ obtained by parallel transporting $v_p$ along the radial geodesic joining $p$ to $q$. So we get a map $X:U\to TU$ which takes $q$ to $v_q$. So $X$ is a vector field on $U$.
Question. Is $X$ necessarily smooth?
What I thought is that by definition of $X$, we have $\nabla_{\partial/\partial r}X=0$ at all point of $U\setminus\{p\}$, where $\partial/\partial r$ is the radial vector field (which is defined at all points of $U$ except $p$). So $X$ is a solution of a system of partial differential equations. I do not know if this guarantees that $X$ is smooth on $U\setminus\{p\}$.