$\DeclareMathOperator{\Spec}{\operatorname{Spec}}$$\DeclareMathOperator{\codim}{codim}$$\Spec(\mathbb{Z})$ cannot be realized as $\Spec(A) \setminus \{m\}$ for a local ring $(A,m)$ - the argument given here can essentially be adapted verbatim. Set $X := \Spec(A)$, $Y := \{m\}$, $U := X \setminus Y \ne \emptyset$. Notice that $\dim U < \infty \iff \dim X < \infty$ - assume this is so. For any $d > 0$, there is an exact sequence (cf. Hartshorne Ex. III.2.3)
$\DeclareMathOperator{\O}{\mathcal{O}}$
$$H^{d-1}(X, \O_X) \to H^{d-1}(U, \O_X \big|_U) \to H^d_Y(Y, \O_X) \to H^d(X, \O_X)$$
Then for $d = \codim Y$, $H^d_Y(Y, \O_X) \ne 0$ by Grothendieck non-vanishing. If $d > 1$, then $H^{d-1}(X, \O_X) = H^d(X, \O_X) = 0$, so $H^{d-1}(U, \O_X \big|_U) \ne 0$, which implies $U$ is not affine (*). This shows that $U$ can be affine only if $d = 1$, in which case $\dim A = 1 \implies \dim U = 0$. Thus the only finite-dimensional rings which can be punctured spectra of local rings are $0$-dimensional.
(*): This is Serre's criterion for affineness: a scheme $X$ is affine iff it is separated, quasi-compact, and $H^i(X, \mathcal{F}) = 0$ for all quasicoherent $\mathcal{F}$, $i > 0$ (see e.g. here).