I think a cool project would be to implement linear algebra in the computer, say for example vector spaces of n-tuples over the rational numbers. You would have to first implement tuples and matrices, then the gaussian reduction algorithm, and then you can implement vector spaces $\Bbb Q^n$, subspaces, perhaps linear maps (unless you identify them with matrices), and operations such as intersection of subspaces, image and kernel of a matrix or linear map, etc.
The reason I suggest linear algebra is because it is a good start if you later on want to continue learning mathematics, and will get you in contact with some fundamental concepts that appear all over mathematics.
Another cool project would be programming the residue class rings $\Bbb Z/m\Bbb Z$. For example $\Bbb Z/6\Bbb Z$ is the set of elements $\{0,1,2,3,4,5\}$ with addition and multiplication performed modulo 6. These objects are finite, and thus allow you to construct small examples and perform experiments. For example you might want to count how many elements $x \in \Bbb Z/m\Bbb Z$ are squares (i.e., exists $y$ with $x=y^2$), and see this for all $m<100$, and maybe find a pattern and guess a formula. You would have to program the elements (represented by an integer between 0 and $m-1$) and their arithmetic operations, and the rings $\Bbb Z/m\Bbb Z$ themselves, then you can program some algorithms, for example the extended Euclidean algorithm for computing multiplicative inverses and solving congruences, etc.