Let $R$ be your relation on $X$. Also, $x\in A$ means that $x$ belongs to the set $A$.
First of all, you do not need to list $(a,a)$ twice as you are describing a set of tuples, i.e $\{(a,a),(a,a)\}=\{(a,a)\}$.
Second of all, in (b), if you want transitivity, then since $(a,b)\in R$ and $(b,c)\in R$ then $(a,c)$ need also be in $R$. This is the same kind of reason for why (c) is also wrong. Since you have $(a,b), (b,a)\in R$ then, by transitivity, you would have $(a,a)\in R$. Also, in (c), since $(a,b), (b,c)\in R$, again by transitivity, you would have $(a,c)\in R$.
Since this sounds like a typical homework problem, I will let you figure out if there is anything more wrong with it. Get back to me if you want, but I suggest that you recap the definitions of reflexive, symmetric and transitive relations.