I'm not sure how well I'm wording this, but I was wondering how you address cases in modular arithmetic when you plug in a number greater than the modulus, e g. $\sqrt{8} \pmod{5}$. If you leave $8$ as it is, and square root it, you get $\approx2.828$. If you mod $8$ before doing anything, you get $\sqrt{3} \pmod{5}$, which is $\approx1.782$ These are clearly different answers, so which one do you go with? Do you mod the input before or after you apply expressions to it?
I hope this question was worded well enough, I was just confused how you would handle that (and I'm sure I'm just misunderstanding something simple here).