every submatrix of such a matrix of size $m$ must have determinant $m!$ or $-m!$. Prove! (observe for such matrix of size $n$ the $n-1$ cofactors must be $(n-1)! $ or $-(n-1)!$ expand along a suitable row or column )
So to prove that there does not exist one such matrix with $n \geq 3$, one just have to prove there cannot be one of size 3. Note that only two possible such matrices of size $2 \times 2$ are $ \left( \begin{array}{cc}
1 & 1 \\
-1 & 1 \end{array} \right)$ and it's transpose. Now a $3 \times 3$ such matrix $A$, after fixing the first row, the second two rows can be covered by three $2 \times 2$ matrices but any such arrangements ends up in two columns being a scalar multiple of each other, thereby contradicting the fact that any $2\times 2$ submatrix has determinant $2 $ or $ -2$. An example below will be:
$ \left( \begin{array}{ccc}
a & b & c \\
1 & 1 & -1 \\
-1 & 1 & 1 \end{array} \right)$ or
$ \left( \begin{array}{ccc}
a & b & c \\
1 & 1 & 1 \\
-1 & 1 & -1 \end{array} \right)$. Hence there cannot be any matix $A$ of only $1, -1$ as entries with size $n$ more than or equal to three with determinant $n!$ ( or -$n!$).