Physicists tend to call the spacetime interval $(\Delta s)^2$ an "inner product," where $(\Delta s)^2=(\Delta t)^2-(\Delta x)^2-(\Delta y)^2-(\Delta z)^2$ up to factors of $c$ and an overall minus sign. But it's not really an inner product, since it breaks positive-definiteness:
$\left< v,v \right> \geq 0$ with equality iff $v=0$.
For that matter, the "metric tensor" of general relativity describes a similar map for two vectors in any tangent space, but this is also not technically a metric (in the mathematical sense) since that would also require positive-definiteness.
So what is the correct mathematical term for this structure?
I would be tempted to call it a pseudo-metric, since spacetime is called a pseudo-Riemannian manifold, but in fact pseudo-metrics still require positivity. I might also expect "pseudo–inner product" to be used, but this phrase does not appear in any Wikipedia article, and the similar term "pseudo-scalar product" is defined here as something completely different.
I have heard the terms "pseudo-Riemannian metric" and "Lorentzian metric," but I doubt mathematicians endorse such terms, since they are of the form "(adjective) + (noun)" where the object is not a type of (noun).