Addressing your comment about, "Why can't $x, y\in\mathbb{C}$?":
The set $S$ of $(x, y)\in\mathbb{R}^2$ which form the circle w/center the origin, and radius 2, has a really interesting property: it's definable, in this case as the zeroes of a polynomial expression. I'm deliberately being vague about what I mean by "definable" - there are a lot of different notions of definability, and they're all interesting. Certainly we shouldn't expect every set of points in the plane to be definable - in fact, if we make things more precise, we can prove that there are undefinable sets of points! EDIT: And, more concretely, we shouldn't even expect every "nice" set of points to be definable in a specific way - say, as the graph of a function. As you correctly say, there is no function whose graph is that circle.
Why is this relevant? Well, let $\varphi$ be the expression "$x^2+y^2=4$." Clearly $\varphi$ defines the set $S$ in $\mathbb{R}^2$ - that is, $S$ is the set of points $(x, y)$ satisfying $\varphi$. However - as you point out correctly - the expression "$x^2+y^2=4$" makes sense in contexts other than $\mathbb{R}^2$. $\varphi$ defines a subset of $\mathbb{C}^2$, and of $\mathbb{Z}^2$, and of $\mathbb{Q}_2^2$ ($\mathbb{Q}_2$ being the really interesting structure called the $2$-adic numbers).
Note that this isn't really all that surprising - the expression "the people with green hair" can be used to define a subset of
the people in this room,
the people in Canada,
the people at this totally awesome Phish concert, or
the rocks which Steve has touched over the last ten years (in this case, not a very interesting subset, but still).
(NOTE: there are some restrictions on where definitions can even be applied. For example, your expression "$x^2+y^2=4$" only makes sense in a context where "$\times$," "$+$," and "$4$" are all meaningful (so it makes sense in an arbitrary ring, but not in an arbitrary group, for example).)
There's an implicit question here - which do I care more about, the set $S$ or the definition (maybe description is a more intuitive word) $\varphi$? One of the most surprising facts, to my mind, of mathematics is:
If you care about some thing $S$, you should also care about the ways $S$ can be defined - and often the best way to understand $S$ is to understand how one of its definitions behaves in other contexts.
The study of definitions as interesting objects in and of themselves is really one of the main driving forces of mathematical logic, and in particular the subfield model theory (http://en.wikipedia.org/wiki/Model_theory). (For me it's the entire motivation, but I don't think that's universal.)
A really striking example of this is something incredibly basic: studying the integer solutions to polynomials with integer coefficients. It turns out that even such basic objects as "natural numbers satisfying some polynomial" hide incredible structure, which is best understood by - among other things - looking at what they would be if "natural numbers" meant something completely different.
And, just to end on a ridiculous note, a really really extreme version of "understanding $S$ by looking at its analogues in other contexts" is this utterly weird and opaque, but brilliant, thing called "interuniversal Teichmuller theory" which was recently discovered/invented by a Japanese mathematician Shinichi Mochizuki. (BTW I don't actually know anything about ITT - my sentence about it is based purely on what I took away from my friend's excited ramblings on the subject.)
We need to be careful, of course: a thing may have different definitions which agree in the context where that thing lives, but disagree in other contexts. For example, the circle $S$ you mention in your question can be described - in the context of $\mathbb{R}$ - as the set of points $(x, y)$ which:
EITHER satisfy "$x^2+y^2=1$,"
OR satisfy "$x^2+y^2=1$, or $x=7, y=-\pi$, and -1 has a square root."
Obviously these expressions define the same subset of $\mathbb{R}^2$, but different subsets of $\mathbb{C}$. This example might strike you as stupid, but this is an important phenomenon which can cause a lot of problems.