The short answer is no.
Each arrow is perpendicular to a tangent line of the curve. A vector pointing in the direction of the tangent line at $x$ is $(1,f'(x))$, so a vector pointing perpendicular to the tangent line is $(-f'(x),1)$. (The dot product of these two vectors is zero.) The sense of the arrows in the picture (up or down) is related to the concavity: pointing up for concave up (convex) and down for concave down (concave). Since $f''(x)\geq 0$ in the former case and $f''(x)\leq 0$ in the latter case, then each arrow is pointing in the direction $$[f''(x)](-f'(x),1).$$ Thus the direction of each arrow depends on both the first and second derivatives (i.e. on both the slope and the concavity).
For example at a local minimum where the function has zero derivative and positive second derivative (convex), the arrow points in the direction $(0,1)$.