There is a problem with your definition of $\infty$.
You can't say that $\infty$ is the number larger than any finite number, because $\infty$ is not a number in the first place. You can't say that some set of numbers ($\mathbb N$ or $\mathbb R$, for example) contains $\infty$ if you want usual rules to hold; for example from $$\infty + a = \infty$$
Subtract $\infty$ to get
$$a = 0$$ for every $a$. This is clearly absurd.
So $\infty$ is not a number, and $\frac 1\infty$ does not mean anything. You can say that
$$\lim_{x\to \infty} \frac 1x = 0$$ but this has a very precise meaning and you should work with that.
This is defined as
$$\forall \epsilon > 0 \exists N: x > N \implies \left|\frac 1x\right| < \epsilon$$
It basically means: for very number greater than zero, if you take numbers that are big enough, eventually $\frac 1x$ will become smaller than the number you've chosen at the beginning.
You can see that this responds very well to what we think of infinity (numbers as big as you want!) but the actual object of $\infty$ is nowhere to be seen.
Note
I have talked about standard analysis. You can define this concepts in a meaningful way (see hyper reals) but again, you have to ask yourself: What am I trying to model? Will this be useful, other than consistent? To be honest, I will advise not to bother with such formulations at the beginning ;-)