This is more of a question in philosophy and semantics. The question of whether or not we define mathematics as a science is not a functionally useful question by itself. Its usefulness arises when we compare and contrast how math and science are related in their methodology and application.
Demarcation
For starters, demarcation of the sciences is a fundamental problem in the philosophy of science, but there is no such analogue in mathematics. There is not a singular answer of what makes something considered "science". Without getting too deep into specifics, there is a gradient of what is considered psuedoscience versus science, which is generally the same metric we use to label "hard sciences" versus "soft sciences", very similar to the differentiation you describe as "natural" versus "social" sciences.
On the other hand, classifying something as math is more straightforward. The only relevant question is whether or not the statement is well-founded under some axiomatic system. (My wording here may not be technical enough, feel free to correct me.)
Methodology
While math can rely on experimental computations as evidence, they never constitute a real proof. Math can rely on hard logical proofs that (under a certain axiomatic system) can be guaranteed to never be wrong (assuming the proof is constructed correctly). We can demonstrate with absolute certainty that there are infinite prime numbers.
Like you mentioned before, science relies on empirical evidence, so we can never use a purely logical proof for anything scientific. We apply mathematical models to our scientific theories because the empirical evidence lines up with it, but we can never be sure that there aren't any exceptions that break under that model. Consider classical mechanics vs quantum mechanics. We used to believe that we can just add velocities linearly, because under every empirical measure it was close enough to being correct. If we treat a set of empirical data as a set of discrete points under some function, there will always be an infinite (and uncountable) number of functions that satisfy those points. Science aims to find the simplest function that models the data and corrects for accuracy over time as new information arises.
To make matters even more difficult, science is rarely (possibly never) a closed system. We can idealize closed systems in order to attempt to understand something about the system, just as how we apply mathematical models to our data, but almost no scientific finding occurs in a truly closed system. Our empirical measurements are always prone to be altered by something outside of the context of our findings. This is not the case in mathematics. We may see an overlap of different fields of math, but it is always self-contained within the axioms, which form a closed system.
Application
Because of the reasons listed above, scientific discoveries are structurally different in application. As findings are produced, we need to update the meta data and conclusions accordingly. This could mean that established findings could be refuted and overturned within the community. Findings that reference those findings would then need to be revised or looked over again. This doesn't really happen much in math unless there is an objective logical error found within a paper. Even the cases where something similar to this happens, the rigor of modern mathematics makes this less and less likely, and often times it could just be working under a different context, which is valid as long as it is self-contained.
Most all sciences are built upon the formal sciences, or math. No matter what field, all results and conclusions need to be founded logically, and the nature of this logic's existence is just accepted, despite its intangibility. There is a debate about whether math was discovered or invented. The school of thought that math objects just inherently exist is called Platonism. The school of thought that math objects are the result of human-invented rules is formalism. If you accept the Platonism position, you could argue that math is a science, as these are observations on existing objects. If you accept the formalism position, then this contradicts the idea that we are studying the natural world, but rather manmade constructs and ideas.
Epistemology
Humans are inherently inductive learners. Learning is a process that requires a balance between hard memorization of previous outcomes and abstracting concepts and generalizations from them. For instance, if you touch a hot surface and feel pain, inductive reasoning would lead you to believe that you would not want to touch other similar hot surfaces. There is no logical reason to assume that other hot surfaces would similarly cause you pain outside of induction. You could argue that scientific studies can help explain the concepts of heat, touch, and pain, but remember that scientific reasoning is also inductive by its definition.
While math is largely a deductive process, humans gather inspiration about how to engage in the process through inductive reasoning. We can find a conjecture that we attempt to approach by some inductive process until we find the deductive proof. In this way, both science and math are the same. You could argue that any field that ever existed similarly requires inductive reasoning, but the goal of science and math is to formalize the inductive process into something more measurable.