At least at my university, there is no course which actually teaches this material. We were presented with axioms for how the real numbers work. I would guess that this is the current trend, at least in my country (UK), but I can't speak for everyone.
First off, construction of the complex numbers is, for some reason, typically left to books which actually discuss some complex analysis. That is to say, excepting a few mentioned by others, I don't know of many books which talk about both the integers and the complex numbers. Some good books which do talk about complex numbers include:
All of the above contain a construction of the complex numbers in the first chapter. I should mention at this point that it might also be worth googling for resources, for example ProofWiki has a construction of the complex numbers. For non-complex number systems ($\mathbb{N},\ldots,\mathbb{R}$), a nice "do-it-yourself" version is included in Terence Tao's Analysis I notes, which can be found at this webpage, and these have also been published in a slightly more refined book format.
A great book which spends a good deal of time focusing on construction of the basic number systems (that is, excluding the complex numbers, as I recall) is:
I guess the main topic of this book could be described as "foundational issues"; hence why construction of numbers plays a prominent role.
On a similar note, introductory set theory textbooks often contain constructions of numbers (often up to and including the real numbers). For example, the following books:
Construction of $\mathbb{Q}$ from $\mathbb{Z}$ is I think covered in Alice in Numberland above. Nevertheless, I'd like to mention here that this construction is standard in abstract algebra: it's called the field of fractions of an integral domain. Constructing $\mathbb{Q}$ is the special case when the integral domain in question is $\mathbb{Z}.$ Two books which I know cover this are:
The references on the Wikipedia page linked to previously must also talk about this, but I haven't looked at those books.
The fact that construction of the reals appears in set theory texts is in part because of the role of set theory as a foundation for mathematics. However, classically, the construction of the reals belongs to real analysis. A construction of the reals is provided, for example, in the following books:
Finally, the classic reference on this whole topic (the progression from $\mathbb{N}$ to $\mathbb{R}$) is:
This last is really dry. There is nothing in this book except for theorems and their proofs. But that might be just what you're looking for.