If you want another perspective on the thing, consider a nonstandard model of Peano arithmetic.
The usual intuition of such things is that a nonstandard model contains the ordinary natural numbers along with a bunch of strange infinite numbers, but nonetheless still manage to just like the ordinary natural numbers, even satisfying the axiom of induction... provided you limit yourself to internal statements: i.e. statements that can be expressed in the language of Peano arithmetic.
You can flip that around, though -- you can imagine the nonstandard model to be the ordinary natural numbers, and the standard model to be some weird smaller collection of natural numbers that is somehow bounded, but yet somehow manages to consistently obey the axioms of Peano arithmetic, so long as you stick to internal, small statements.
My intention, of course, is that the small natural numbers -- i.e. the ones contained in the standard model -- now corresponds more closely to "the subclass natural numbers accessible to humans". And you have added features; e.g. this arrangement doesn't require there to exist a largest small natural number. "Small" isn't part of the language of Peano arithmetic, so the usual inductive proof that you would have to have such a thing doesn't apply.
Interestingly, I believe it is possible, in such a setup, to develop an "infinite set theory of small sets". As the first step, you can use large sets as a proxy for proper classes; two large sets represent the same class iff they have the same small elements. Then, you can use proper classes as objects and go up to the next order. I'm not entirely sure how far you can get with such a programme.
Alas, the only sort of axiomatization of this sort of thing I've seen is "internal set theory", which is based on ZFC rather than Peano arithmetic, but it may still appeal to you.
Writing this answer has really helped me clarify my opinions on ultrafinitsm. I want to say that the "human-accessible" numbers already Peano axioms directly; no need to try and come up with a new axiomatization.
What we need instead is a larger theory that lets us talk about the possibility that there are numbers inaccessible to humans. Thus, my focus on embedding the human-accessible numbers as a standard model inside a larger non-standard model (or maybe I should coin new terminology: to embed the human-accessible natural numbers as a substandard model of the natural numbers). And I want to do this in a way that defeats the usual 'weirdness' arguments, such as "there must be a smallest number that is human-inacessible".
But as a caveat, most of my knowledge of ultrafinitism comes from the popular controversial accounts, which may have been more focused on claims of the deficiency of standard mathematics rather than exposition about what ultrafinitism actually is.