Looks like you have correct set up for cylindrical coordinates.
When you integrate remember that $\sqrt{R^2 - R^2\cos^2\theta} = R|\sin\theta|$
In spherical:
$x = \rho\cos\theta\cos \phi\\
y = \rho\cos\theta\cos \phi\\
z = \rho\sin \phi$
you will have to break it up into two integrals.
For one integral $\rho$ will have the freedom to move from $0$ to $R$
But we need to find limits for $\phi$
Lets find the contour where the cylinder meets the sphere
$(x,y,z) = (R\cos^2\theta, R\cos\theta\sin\theta, R\sin\theta)$
$\phi = \theta$
$2\int_{-\frac{\pi}{2}}^\frac{\pi}{2}\int_\theta^\frac{\pi}{2}\int_0^R \rho^2\sin\phi\ d\rho\ d\phi\ d\theta$
Since the region is symmetric across they xy plane, I am integrating half the region and doubling it.
Now we need the integral where $\rho$ is limited by the cylinder wall.
plugging the polar expressions into
$x^2 + y^2 - Rx = 0$ we get
$\rho^2\cos^2\phi - R\rho\cos\theta\cos\phi = 0\\
\rho = R\cos\theta\sec\phi$
$2\int_{-\frac{\pi}{2}}^\frac{\pi}{2}\int_0^\theta\int_0^{R\cos\theta\sec\phi} \rho^2\sin\phi\ d\rho\ d\phi\ d\theta $