First, let's dispense with the trivial example: given a statement $\varphi$, let $P_\varphi$ be the problem "What is the least number which is $0$ if $\varphi$ is true, and $1$ if $\varphi$ is false?" Then $P_\varphi$ is a question about a single finite object, but is equivalent to solving $\varphi$.
Of course, this is a silly example; can we do better?
One approach to this is to look at the quantifier structure. Suppose I have a problem expressed in the form "What is the least $n$ such that [stuff]" - I want to ask, How complicated is [stuff]? This leads naturally to the arithmetic hierarchy, which lets us organize problems according to their intrinsic complexity. For instance, the Twin Prime conjecture is at level $\Pi^0_2$: "For all $n$, there exists $m$ such that $m>n$ and $m, m+2$ are each prime." Meanwhile, the version of it that you state (finding the minimal recurrent gap) is asking about the minimal element of a $\Pi^0_2$-definable set (the set of $k$ such that for all $n$, there is an $m>n$ with $m, m+k$ each prime), and the question of whether there is any recurrent gap is $\Sigma^0_3$.
There is a computability-theoretic connection here. If we let $T_n$ be the set of true $\Sigma^0_n$ sentences, it turns out that $T_n$ is strictly less complicated than $T_{n+1}$: $T_1$ is (basically) the Halting Problem, and $T_{n+1}$ is the Halting Problem "relative to" $T_n$.
Meanwhile, let's attack the stronger question you ask: whether every problem can be reduced to something "finitary." Of course the question is vague, but I would argue the answer is no - for overkill, we could look at the Continuum Hypothesis as an example of a statement which I don't believe can be reduced to a question about finite objects in any nice way. This can in fact be made precise: since CH is not invariant under forcing, Shoenfield absoluteness implies that it is not equivalent to any $\Pi^1_2$ sentence, and I'd argue that $\Pi^1_2$ strictly contains the "finitary" sentences by a wide, wide margin.
You may also be interested in this recent question.