Let $AB$ be the diagonal of the square that is regarded as a arc of an infinite circle, and let $C$ be the corner of the square that is the centre of the first circle:

In the general case, if the height of the circular segment $ADBF$ cut off by the chord $AB$ is $h,$ then
$$
r - \sqrt{r^2 - \frac{a^2}2} = h,
$$
therefore
$$
r + \sqrt{r^2 - \frac{a^2}2} = \frac{a^2}{2h},
$$
therefore
$$
r = \frac12\left(h + \frac{a^2}{2h}\right) = \frac{2h^2 + a^2}{4h}.
$$
Check: if
$$
h = a - \frac{a}{\sqrt2},
$$
then $2h^2 = (3 - 2\sqrt2)a^2,$ therefore
$$
r = \frac{(4 - 2\sqrt2)\sqrt2a^2}{4(\sqrt2 - 1)a} = a. \ \checkmark
$$
In the case of interest,
$$
h = \frac12\left(a - \frac{a}{\sqrt2}\right) = \frac{(\sqrt2 - 1)a}{2\sqrt2},
$$
therefore
$$
2h^2 = \frac{(3 - 2\sqrt2)a^2}4,
$$
therefore
$$
r = \frac{(7 - 2\sqrt2)2\sqrt2a^2}{4\cdot4(\sqrt2 - 1)a} = \frac{7 - 2\sqrt2}{4(2 - \sqrt2)}a = \frac{10 + 3\sqrt2}8a \bumpeq 1.7803a.
$$
More neatly:
$$
\frac{r}{a} = \frac12\left(\frac{h}{a} + \frac{a}{2h}\right).
$$
Check: if
$$
\frac{h}{a} = 1 - \frac1{\sqrt2},
$$
then
$$
\frac{a}{h} = 2\left(1 + \frac1{\sqrt2}\right),
$$
therefore
$$
\frac{r}{a} = \frac12\left(1 - \frac1{\sqrt2}\right) +
\frac12\left(1 + \frac1{\sqrt2}\right) = 1.\ \checkmark
$$
In the case of interest,
$$
\frac{h}{a} = \frac12\left(1 - \frac1{\sqrt2}\right),
$$
therefore
$$
\frac{a}{h} = 4\left(1 + \frac1{\sqrt2}\right),
$$
therefore
$$
\frac{r}{a} = \frac14\left(1 - \frac1{\sqrt2}\right) +
\left(1 + \frac1{\sqrt2}\right) =
\frac14\left(5 + \frac3{\sqrt2}\right) = \frac{10 + 3\sqrt2}8.
$$