A primary ideal is a proper ideal by its definition. Any proper nonzero ideal of $\mathbb{Z}$ is of the form $k\mathbb{Z}$ with $k>1$.
⇒ Suppose $k\mathbb{Z}$ is a primary ideal of $\mathbb{Z}$. We need to show that $k\mathbb{Z}=p^n\mathbb{Z}=(p^n)$.
Since $\mathbb{Z}$ is a primary ideal, for any $ab∈\mathbb{Z}$ if $a∉\mathbb{Z}$ then $b^m∈k\mathbb{Z}$ for some positive integer $m$.
Since $k>1$, the prime decomposition of $k$ contains at least one factor, say $p^n$, where $p$ is prime not equal to $1$ and $n≥1$.
Suppose the prime decomposition of $k$ contains two factors, that is, $k=q^α p^n$, where $q,p$ are distinct primes and $α,n∈\mathbb{Z}^+$. Let $a=q^α p^{n-1}$ and $b=p$. Then $ab=q^αp^{n-1}p=q^α p^n=k∈k\mathbb{Z}$ and $a∉k\mathbb{Z}$ (since $k=q^α p^n∤q^α p^{n-1}=a$). And $b^m=p^m$ hence $k=q^α p^n∤p^m=b^m$, therefore $b^m∉k\mathbb{Z}$ for any positive integer $m$. So we constructed an element $ab∈k\mathbb{Z}$ of a primary ideal $k\mathbb{Z}$ such that $a∉k\mathbb{Z}$ and also $b^m∉k\mathbb{Z}$ for any positive integer $m$, a contradiction to the definition of a primary ideal.
(If the prime decomposition of $k$ contains more than two factors, then $k=(\prod_{i}q_i^{α_i})p^n$ and letting $a=(\prod_{i}q_i^{α_i})p^{n-1}$ and $b=p$ we reach the same contradiction just substituting "$q^α$" with "$(\prod_{i}q_i^{α_i})$" throughout the argument).
⦁ Thus, the prime decomposition of $k$ cannot contain more than one factor. And, at the same time, it contains the factor $p^n$. Hence it can only be that $k=p^n$, so that $k\mathbb{Z}=p^n \mathbb{Z}=(p^n)$.
⇐ Suppose $k\mathbb{Z}=p^n\mathbb{Z}=(p^n)$. We need to show that $(p^n)$ is a primary ideal of $\mathbb{Z}$.
Take any $ab∈p^n\mathbb{Z}$ such that $a∉p^n\mathbb{Z}$. It suffices to show that there is some positive integer $m$ such that $b^m∈p^n\mathbb{Z}$. Since $a∉p^n\mathbb{Z}$, it follows that $p^n∤a$, hence the prime decomposition of $a$ either doesn’t contain this prime $p$ or contains a factor $p^{l_1}$ with $1≤l_1<n$, that is, the prime decomposition of $a$ contains a factor $p^{l_1}$ with $0≤l_1<n$. Then, since $k∣ab$, the prime decomposition of $b$ contains a factor $p^{l_2}$ with $l_2≥n-l_1$, so that $n≥l_1+l_2$ and hence $p^n∣p^{l_1+l_2}$. We definitely can take some $m∈\mathbb{Z}^+$ such that $ml_2≥n$. Then, raising $b$ to this power $m$, we get $b^m$, whose prime decomposition contains a factor $p^{ml_2}$, so that $p^n∣b^m$ and hence $b^m∈p^n\mathbb{Z}$.
⦁ Thus, $(p^n)$ is a primary ideal of $\mathbb{Z}$.