For $n=3$ the sums of rows, columns, and wrap-around diagonals determine the matrix, even over the reals, since if we define one sum as that of all the entries, and then two each of rows and columns, and two each of left. respectively right diagonals this is nine equations, whose determinant is $27.$ Note that e.g. there is no need to include the sum of the third row since it is a consequence of the overall sum and that of the other two rows.
For $n=4$ there is the following example built on block $2 \times 2$ matrices. Let $I$ be the (2 by 2) identity, $J$ the complement of that (i.e. rows $0,1;1,0$). Also let $Z$ be the zero matrix and $M_1$ the matrix of all $1$'s.
Form matrix $A$ with block rows $I,Z;M_1,J$ and matrix $B$ the same as matrix $A$ but with the positions of blocks $I,J$ switched.
Then for each of matrices $A,B$ the sums across are $1,1,3,3$ and down are $3,3,1,1$ while all the wraparound diagonals of either $A$ or $B$ have sum $2.$
Added: for convenience here are the two matrices $A,B.$
$$A=\begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\
1 & 1 & 0 & 1 \\ 1 & 1 & 1 & 0 \end{pmatrix} $$
and
$$B=\begin{pmatrix} 0 & 1 & 0 & 0 \\ 1 & 0 & 0 & 0 \\
1 & 1 & 1 & 0 \\ 1 & 1 & 0 & 1 \end{pmatrix} $$
Another interpretation: If by the diagonals the OP does not mean for them to "wrap around", so that they stop where they hit the edges of the matrix, then the "cutoff" is still $n=3$. Two matrices of size $3 \times 3$ with all the corresponding row or column or diagonal sums equal are easily seen to be the same entry by entry, as if they are [a,b,c;d,e,f;g,h,i] and [a',b',c';d',e',f';g',h',i'] then the equality of corresponding length 1 diagonals gives a=a',c=c',g=g',i=i', and then equality of corresponding row sums implies b=b' and h=h', also equality of corresponding column sums gives d=d' and f=f', and finally equality of say the corresponding center row sums gives the remaining e=e'.
For $n=4$ there are the following two matrices $C,D$ having all the corresponding sums of rows, columns, and non-wrapped diagonals equal.
$$C=\begin{pmatrix} 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 \\
1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix}$$
and
$$D\begin{pmatrix} 0 & 0 & 1 & 0 \\ 1 & 0 & 0 & 0 \\
0 & 0 & 0 & 1 \\ 0 & 1 & 0 & 0 \end{pmatrix}$$
For each of these matrices, all row or column sums are $1$, while corresponding "non-wrapped" diagonals have the same sums, specifically the length 1 diagonals have sum $0$, the length 2 or 3 diagonals have sum $1$, and the length 4 diagonals all have sum $0.$
PS: I noticed later that matrices $C,D$ also work as examples where corresponding rows, columns, and wrap-around diagonals all have the same sum. (Making the above example of $A,B$ unnecessary...)