Although there are many answers at this point, I'm compelled to answer because the original question did not use a quantifier, so depending on the logic it may be difficult to use $\exists!$ directly. For example, the statement "exactly one of the sets $x,y,z$ is blue" can be written "$\exists!n\in\{x,y,z\}\, n$ is blue", but "exactly one of the following propositions is true: the moon is made of cheese, I know how to swim, there is an elephant in the White House" is much harder to write in this manner, because there are no sets available to quantify over (propositions can not necessarily be treated as sets; the difference is what lead to Russell's paradox).
If $P,Q,R$ are propositional variables, then "exactly one of $P,Q,R$ is true" can be faithfully represented as:
$$(P\land\lnot Q\land\lnot R)\lor(\lnot P\land Q\land \lnot R)\lor(\lnot P\land\lnot Q\land R)$$
$$(P\oplus Q\oplus R)\land\lnot(P\land Q\land R)$$
$$((P\oplus Q)\land\lnot R)\lor(\lnot P\land\lnot Q\land R)$$
where $P\oplus Q=(P\land\lnot Q)\lor (\lnot P\land Q)$ is the exclusive or.
The third example actually generalizes well; if $\phi_k^n$ means "exactly $k$ of $P_1,P_2,\dots,P_n$ is true" then these formulas can be defined inductively as
$$\phi_0^1=\lnot P_1\quad\phi_1^1=P_1\qquad\phi_0^n=\phi_0^{n-1}\land\lnot P_n\quad\phi_n^n=\phi_{n-1}^{n-1}\land P_n\qquad$$
$$\phi_k^n=(\phi_{k-1}^{n-1}\land P_n)\lor (\phi_k^{n-1}\land\lnot P_n)$$
although the length of the formula is still quite long, ${n+2\choose k+1}-2$ in general, and I don't know if there is an asymptotically shorter formula.
One last special case: The number one case (in fact, the only case) where I've seen this propositional "exactly one" occur in math, and which may be the prototype from which you abstracted this question, is in the statement of the trichotomy law for real numbers:
If $x,y$ are real numbers, then exactly one of $x<y$, $x=y$, $x>y$ is true.
In this particular case (in the presence of antisymmetry), it turns out that you can rewrite this law into a simpler one using only the biconditional:
If $x,y$ are real numbers, then $\lnot\, x<y\leftrightarrow(x=y\lor y<x)$.
It is a good exercise to show that this axiom implies both trichotomy and antisymmetry.
xorI proposexoneorxorne– Tobias Kienzler Mar 10 '15 at 12:43