EDIT: My previous answer gave an argument which only showed that a map $X \to S^1 \vee \mathbb{R}P^3$ is nullhomotopic. Thanks to @Jason Devito for pointing it out. I planned to delete this incorrect answer, but I had some other thoughts which I offer as a mea culpa below.
One way to approach the problem is to note that the space $X$ is the cofiber (mapping cone) of a map $S^1 \vee S^1 \to S^1 \vee S^1$ given by "$(a^3, (ab)^5)$". Therefore, there is a long exact sequence of pointed sets $$\cdots \to [S^2 \vee S^2, Y] \xrightarrow{\text{(ii)}} [S^2 \vee S^2, Y] \to [X, Y] \to [S^1 \vee S^1, Y] \xrightarrow{\text{(i)}} [S^1 \vee S^1, Y].$$ In order to show that $[X,Y] = \{*\}$, we would need to show that the map labelled by (i) is injective and the map labelled by (ii) is surjective.
That (i) is injective follows from the algebraic manipulations alluded to in the OP. The map (i) takes a pair $(a, b)$ of elements in $\pi_1 Y$ to the pair $(a^3, (ab)^5)$. Since $\pi_1 Y = \mathbb{Z} * \mathbb{Z}/2$, the relation $(a^3, (ab)^5) = (1,1)$ forces $a = b = 1$, so (i) is injective.
For (ii), we have $\pi_2 Y \cong \mathbb{Z}$, but the map $\mathbb{Z}^2 \to \mathbb{Z}^2$ given by $(m,n) \mapsto (3m, 5(m+n))$ is not surjective. So this would suggest that $[X,Y] \neq \{*\}$.