If we want to make the question more interesting by allowing pixels with the same colour (i.e. indistinguishable pixels), the general solution can be obtained as follows:
Let $n_c$ be the number of colours and $c_i, i=1..n_c$ be the multiplicities of the colours. Then the total number of permutations is given by
$$P = \frac{(xy)!}{\prod_{i=1}^{n_c} c_i!}$$
This is because there are $c_i!$ ways to rearrange the $i$-coloured pixels without changing the image. To get the denominator in a different form, we can use
$$d(n):=|\{i | c_i \ge n\}|$$
wich counts how many colours $i$ have at least $n$ pixels coloured by $i$. Then the denominator is
$$\prod_{i=1}^{n_c} c_i! = 1\cdot2\cdot \ldots c_1 \cdot 1 \cdot \ldots c_2 \ldots c_{n_c} = 1\cdot 1\cdot \ldots 1 \cdot 2 \cdot \ldots = \prod_{j=1}^{xy} j^{d(j)}$$
The upper limit of the RHS product can be replaced by $\max_{i} c_i \le xy$
How would I go about doing this with a 5000x5000 pixel square with 255 colors?
– Martin Erlic Dec 03 '14 at 20:11