For natural integers $\{0,1,2,\ldots\}$, subtraction is defined as removal indeed.
But in that case you can't define $3-5$. You can't remove $5$ from $3$.
But what you can do is to define $3-5$ as the operation of adding $3$ then removing $5$ to something. For instance $1000 + 3 - 5 = 998$.
But this operation is equivalent to adding $2$ and then removing $4$, because $1000+2-4 = 998$ also. So $3-5 = 2 -4$. It is also the equivalent to adding nothing and removing $2$. So $3-5 = 0-2$, which is pure removal, which we denote by the shorthand $-2$ (forgetting the leading $0$).
In general, we say that $a-b$ is the same as $-d$ if $a+d = b$.
This is how mathematicians construct the relative integers $\{\ldots,-2,-1,0,1,2,\ldots\}$. This is explained in the wikipedia page about the integers, where you can find good sources such as Campbell, Howard E. (1970). The structure of arithmetic. Appleton-Century-Crofts. p. 83
I think it is pretty clear how this applies to your example. A variation of population is, by design, a relative integer (it can be negative, population can decrease). It corresponds to adding a certain number of people (the births) and removing another (the deaths). Hence your equation.