I've been thinking recently about some nice moduli problems in algebraic geometry as well as the relationship of moduli spaces to string theory, gauge theory, and such. Mathematically, to my understanding, we first find some parameter space for certain objects, and then quotient by whatever we want to identify, passing us to the moduli space. Physicists speak of perhaps having some initial parameter space, or "theory space" and then identifying certain points based on symmetries or dualities. It seems like there's a slew of hugely beautiful examples where mathematically what we quotient by to pass to the moduli space corresponds precisely to the same purely physical identification, giving us an exact correspondence to the moduli space. The main example that comes to mind is the elliptic curve: whenever an elliptic curve appears, it seems like there's a purely physical reason to identify two "theories" which are exchanged by $\tau \to -1/\tau$.
So my question is, are there examples (either in pure maths, or at the interface of math/physics) where something corresponds to the parameter space of some class of objects, but not the proper moduli space itself? Or perhaps, I should ask, if one comes across such a scenario, what is this hinting at? Is it hinting that the two things are not really connected in any nice way? Again, maybe as a simple example, one might have something resembling an elliptic curve which is invariant under $\tau \to \tau + 1$, but not $\tau \to -1/\tau$. Are there examples where a nice correspondence can still exist between two such things, or is this a sign that the two are not really connected?