First, substitute via $u=1+\ln^2(x)$ and use the complex exponential form of cosine to get
\begin{align*}
\int \ln(x) \cos(1+\ln^2(x)) \,dx
&= \int \sqrt{u-1}\, \cos(u)\, \frac{\exp(\sqrt{u-1})}{2\sqrt{u-1}} \,du \\
&= \frac{1}{2} \int \cos(u)\,\exp(\sqrt{u-1}) \,du\\
&= \frac{1}{2} \int \frac{1}{2}[\exp(iu)+\exp(-iu)] \,\exp(\sqrt{u-1}) \,du \\
&= \frac{1}{4}
\left[ {\int \exp(iu) \,\exp(\sqrt{u-1}) \,du}
+
\int \exp(-iu) \,\exp(\sqrt{u-1}) \,du
\right]\\
&= \frac{1}{4}
\left[ I_1 + I_2 \right]
\end{align*}
Let's deal with $I_1$ first.
\begin{align*}
I_1
&= {\int \exp(iu) \,\exp(\sqrt{u-1}) \,du}\\
&= \underbrace{-\exp(\sqrt{u-1}) i \exp(iu) }_{\gamma_1}
- \int -i\exp(iu)\frac{\exp(\sqrt{u-1})}{2\sqrt{u-1}} \,du\\
&= -\gamma_1 + ie^i\int \exp(a^2i+a) \,da\\
&= -\gamma_1 + ie^i\int \exp\left( i\left[a-\frac{i}{2}\right]^2 +\frac{i}{4} \right) \,da\\
&= -\gamma_1 + i\exp\left(\frac{5i}{4}\right)\int
\exp\left( \left[a\sqrt{i} - \frac{i^{3/2}}{2} \right]^2 \right) \,da\\
&= -\gamma_1 + \sqrt{i}\exp\left(\frac{5i}{4}\right)\int
\exp\left( z^2 \right) \,da\\
&= -\gamma_1 + \frac{\sqrt{i\pi}}{2}\exp\left(\frac{5i}{4}\right)\,\text{erfi}(z)
\end{align*}
where the first step uses integration by parts, then the substitution
$a=\sqrt{u-1}$ is used, followed by the substitution $z=a\sqrt{i} - i^{3/2}/2$.
Recall that $\text{erfi}(z)=-i\,\text{erf}(iz)$.
Now for $I_2$, using a similar strategy.
\begin{align*}
I_2
&= {\int \exp(-iu) \,\exp(\sqrt{u-1}) \,du}\\
&= \underbrace{\exp(\sqrt{u-1}) i \exp(-iu)}_{\gamma_2}
- \int i\exp(-iu)\frac{\exp(\sqrt{u-1})}{2\sqrt{u-1}} \,du\\
&= \gamma_2 -
i\int \exp(-ia^2-i+a) \,da\\
&= \gamma_2 -
ie^{-i}\int \exp\left( -i\left[a + \frac{i}{2}\right]^2 -\frac{i}{4} \right) \,da\\
&= \gamma_2 -
i \exp\left( \frac{-5i}{4} \right)
\int \exp\left( -\left[ a\sqrt{i} + \frac{i^{3/2}}{2} \right]^2 \right) \,da\\
&= \gamma_2 -
\sqrt{i} \exp\left( \frac{-5i}{4} \right)
\int \exp\left( -\zeta^2 \right) \,da\\
&= \gamma_2 -
\frac{\sqrt{i\pi}}{2}
\exp\left( \frac{-5i}{4} \right) \,
\text{erf}(\zeta)\\
\end{align*}
using the same subsitution with $a$ and with $\zeta = a\sqrt{i} + i^{3/2}/2$.
Ok, now let's simplify the terms without the error functions:
\begin{align*}
-\gamma_1 + \gamma_2 &=
-\exp(\sqrt{u-1}) i \exp(iu) +
\exp(\sqrt{u-1}) i \exp(-iu) \\
&=
-\underbrace{\exp(\sqrt{u-1})}_{x}i[\underbrace{e^{iu} - e^{-iu}}_{2i\sin(u)}] \\
&=
2x\sin(1+\ln^2(x)) \\
\end{align*}
where we used the complex exponential form of sine.
Next we need to be able to undo the substitutions:
\begin{align*}
z
&= \sqrt{i} a - \frac{i^{3/2}}{2}\\
&= \sqrt{i} \sqrt{u-1} - \frac{i^{3/2}}{2}\\
&= \sqrt{i} \ln(x) - \frac{i^{3/2}}{2}\\
&= \frac{1}{2} \sqrt{i}\left[ 2\ln(x) - i \right]\\
\zeta
&= \sqrt{i} a + \frac{i^{3/2}}{2}\\
&= \sqrt{i} \sqrt{u-1} + \frac{i^{3/2}}{2}\\
&= \sqrt{i} \ln(x) + \frac{i^{3/2}}{2}\\
&= \frac{1}{2} \sqrt{i}\left[ 2\ln(x) + i \right]
\end{align*}
Now we can put it all together:
\begin{align*}
\int & \ln(x) \,\cos(1+\ln^2(x)) \,dx \\
&= \frac{1}{4} \left[ I_1 + I_2 \right]\\
&= \frac{1}{4} \left[
-\gamma_1 + \frac{\sqrt{i\pi}}{2}\exp\left(\frac{5i}{4}\right)\,\text{erfi}(z)
+
\gamma_2 -
\frac{\sqrt{i\pi}}{2}
\exp\left( \frac{-5i}{4} \right)
\text{erf}(\zeta)
\right]\\
&=
\frac{1}{4} \left(
-\gamma_1 + \gamma_2 +
\frac{\sqrt{i\pi}}{2}
\exp\left(\frac{-5i}{4}\right)
\left[
\exp\left( \frac{5i}{2} \right)
\text{erfi}(z)
-
\text{erf}(\zeta)\right]
\right)\\
&=
\frac{1}{4} \left(
2x\sin(1+\ln^2(x)) +
\frac{\sqrt{i\pi}}{2}
\exp\left(\frac{-5i}{4}\right)
\left[
\exp\left( \frac{5i}{2} \right)
\text{erfi}(z)
-
\text{erf}(\zeta)\right]
\right)\\
&=
\frac{1}{8} \left(
4x\sin(1+\ln^2(x)) +
{\sqrt{i\pi}}
e^{\frac{-5i}{4}}
\left[
e^{\frac{5i}{2}}
\text{erfi}\left(\frac{\sqrt{i}\left[ 2\ln(x) - i \right]}{2} \right)
-
\text{erf}\left(\frac{\sqrt{i}\left[ 2\ln(x) + i \right]}{2} \right)\right]
\right)
\end{align*}
A quick check in Mathematica gives:
f[x_] := Log[x] Cos[1 + (Log[x])^2]
FullSimplify[Integrate[f[x], x]]
1/8 ((-1)^(1/4) E^(-((5 I)/4))
Sqrt[\[Pi]] (-Erf[1/2 (-1)^(1/4) (I + 2 Log[x])] +
E^((5 I)/2) Erfi[1/2 (-1)^(1/4) (-I + 2 Log[x])]) +
4 x Sin[1 + Log[x]^2])
Which maybe should have been the whole answer...
See also this question, which solves a similar problem.