As pointed out in the comments, the integral is non-elementary i.e. the integrand doesn't possess a primitive. However, you can compute it to arbitrary accuracy using the expansion for $\cos(x)$. We have
$$\cos(x) = \sum_{k=0}^{\infty} \dfrac{(-1)^k x^{2k}}{(2k)!}$$
Hence,
\begin{align}
\int_0^t \log(x) \cos(x) dx & = \int_0^t \log(x) \left(\sum_{k=0}^{\infty} \dfrac{(-1)^k x^{2k}}{(2k)!} \right) dx\\
& = \sum_{k=0}^{\infty} \dfrac{(-1)^k}{(2k)!} \int_0^t x^{2k} \log(x) dx\\
& = \sum_{k=0}^{\infty} \dfrac{(-1)^k}{(2k)!} \dfrac{t^{2k+1} \left((2k+1) \log t - 1\right)}{(2k+1)^2} \,\,\,\,\, (\spadesuit)
\end{align}
Note that, in general, you need to argue out why you can swap the integral and infinite sum. In this case, this is not hard since the integral is dominated by $\displaystyle \int_0^t \vert \log(x) \vert dx$.
Now truncating the infinite series $(\spadesuit)$ will give you arbitrarily accurate answer. Also note that, for a fixed $t$, the series converges exponentially. Hence, a good approximation is $$\sum_{k=0}^{n} \dfrac{(-1)^k}{(2k)!} \dfrac{t^{2k+1} \left((2k+1) \log t - 1\right)}{(2k+1)^2}$$ for reasonably large $n \gg t$.