Suppose $S$ is a countable, dense subset of $[0,1]^2$ and $f = \chi_S$. Here $f$ is not Riemann integrable on $[0,1]^2$ since, given any partition $P$, each rectangle contains points where $f(x) = 1$ and $f(x) = 0$ and the upper Darboux sum $U(P,f) = 1$ while $L(P,f) = 0$.
However, we can construct a countable, dense subset $S$ which intersects each vertical and horizontal line in at most one point and, consequently, both iterated integrals exist and are equal to $0$. With such a set $S$, for any $x \in[0,1]$ there is only one point $(x,y_x)$ such that $f(x,y_x) = 1$, but $f(x,y) = 0$ if $y \neq y_x$. Thus,
$$\int_0^1 f(x,y) \, dy = 0 \implies \int_0^1 \left(\int_0^1 f(x,y) \, dy \right)\, dx = 0,$$
with a similar argument and result for the other iterated integral.
Construct $S$ by starting with any countable, dense subset $\{(x_j,y_j): j \in \mathbb{N}$} and then choosing
$$(\hat{x}_1,\hat{y}_1) = (x_1,y_1), \\ (\hat{x}_2,\hat{y}_2) \in B((x_1,y_1), 1/2) \text{ with }(\hat{x}_2,\hat{y}_2) \neq (x_1,y_1), \\ (\hat{x}_3,\hat{y}_3) \in B((x_3,y_3), 1/3) \text{ with }(\hat{x}_3,\hat{y}_3) \neq (x_1,y_1), (x_2,y_2), \\ \ldots $$
The linked answer in the comment provided by @onurcanbektas uses the function
$$f(x,y) = \frac{x^2 - y^2}{(x^2+y^2)^2}$$
This serves better as an example where the conclusion of Fubini's theorem is false for Lebesgue integrals because $|f|$ is not integrable. In the context of Munkres' question, $f$ in this case fails to be Riemann integrable simply because it is unbounded. (The hint leads to an example where the function is bounded.) Another technicality arises in consideration of the unequal iterated integrals as Riemann integrals. For example,
$$\int_0^1 \frac{x^2 - y^2}{(x^2 + y^2)^2} \, dy = \begin{cases} \frac{1}{1+x^2}, & x \neq 0 \\ -\infty, & x = 0 \end{cases}$$
where $\int_0^1f(0,y)\, dy$ does not exist as a Riemann integral. Of course, we can bypass this issue by recognizing that the non-existence occurs on a set of measure zero and either accepting $\pm \infty$ as extended real numbers or reassigning the value.