For $p(v)=\inf\{r>0\mid rv\notin C\}$, if $\mathbb R\setminus C$ contains a neighborhood of $0$, then $p(v) = 0$ for all $v$, which is not very instructive.
For fixed $v$, let $P = \{r>0\mid v/r \in C\}$ and $P^* = \{r>0\mid rv\in C\}$.
If $r>0$, we have
$$r \in P^* \iff rv \in C \iff \frac {v}{1/r} \in C \iff \frac 1r \in P$$
It follows that
$$\sup P^* = \frac1{\inf P},$$
where the 'equality' $\infty = 1/0$ is implicit.
Hence, with $p^*(v) = \sup\{r>0\mid rv\in C\}$, we'd have $p^*(v) = 1/p(v)$.
I guess what you should ask yourself at this point is: do you prefer to work with infinites, or with $0$s?
If you want a more prosaic answer (but probably also more correct), I'd just say that people have worked with $p(v)$ and found it to have nice properties that might not be immediately or as easily intuitive (at least notationally) when working with $1/p(v)$.
In particular, $p$ can be equivalently written as
$$p(v) = \inf \{r>0\mid v \in rC\},$$
which may now remind you of Minkowski functionals.
Under certain conditions $($on $C)$, these functionals have very nice properties: they are a norm for which $C$ is a ball!