Sometimes, a topological space has indistinguishable points - we call those spaces non-$T_0$. But given such a space, we can always identify indistinguishable points, thereby yielding a $T_0$ space. (Technically, we've taken the Kolomogorov quotient).
Does this sort of thing ever happen in abstract algebra?
Here's two more examples.
- A preordered set can have comparable, distinct points - in other words it can fail to be antisymmetric. But that's cool, we can identify comparable points to obtain a partially ordered set.
- Sometimes a pseudometric space has distinct points that are zero distance apart. But that's okay, we can identify zero-distance points to obtain a metric space.
Edit: It would be nice to see a definition of 'indistinguishable' for the elements of arbitrary structures. It would then be a consequence of this more general definition that for an arbitrary preordered set $X$ (order relation $\leq$) it holds that $x,y \in X$ are indistinguishable iff $x \leq y$ and $y \leq x$.
Here's an example. Consider the function $f : \mathbb{N} \rightarrow \mathbb{N}$, with $f(n)=0$ for all $n \in \mathbb{N}$. The associated notion of indistinguishability for the structure $(\mathbb{N},f)$ should probably be the relation $\sim$ such that $a \sim b$ iff both $a$ and $b$ equal $0$, or both $a$ and $b$ are distinct from $0$.
Edit2: On the other hand, perhaps it does not make sense to speak of 'the natural notion of indistinguishability in a structure $X$' without first situating that structure in a category. After all, if we're going to quotient out by the indistinguishability relation, epimorphisms will probably show up at some point.