I know I'm super late to the party but I got stuck same as you and I just want a record of my solution in case I forget. First, to simplify the notation a little bit, define $I := I_{(j,\alpha)}^k$. Then you're trying to compute
$$\int_{[0,1]^{k-1}} I^* (f \; dx^1 \wedge \cdots \wedge \widehat{dx^i} \wedge \cdots \wedge dx^k).$$
Now let $p = (x_1,x_2, \ldots, x_{k-1})$.This implies
$$I(p) = (x_1,\ldots, x_{j-1},\alpha,x_j,\ldots,x_{k-1})$$
Then we know
$$I^* (f \; dx^1 \wedge \cdots \wedge \widehat{dx^i} \wedge \cdots \wedge dx^k)=(f\circ I) \; I^*(dx^1 \wedge \cdots \wedge \widehat{dx^i} \wedge \cdots \wedge dx^k)$$
Now $$f\circ I(p) = f(x_1,\ldots, x_{j-1},\alpha,x_j,\ldots,x_{k-1})$$
And
$$I^*(dx^1 \wedge \cdots \wedge \widehat{dx^i} \wedge \cdots \wedge dx^k)(p)(v_1,\ldots, v_{i-1},v_{i+1},\ldots,v_{k})\\
=
(dx^1 \wedge \cdots \wedge \widehat{dx^i} \wedge \cdots \wedge dx^k)(I(p))(I_*(v_1),\ldots, I_*(v_{i-1}),I_*(v_{i+1}),\ldots,I_*(v_{k}))$$
To make the notation cleaner, we'll drop the $I(p)$ keeping in mind that our $dx^i$ are duals to $T_{I(p)}\mathbb{R}^n$
$$I^*(dx^i)(v_{i}) = dx^i \left(
\sum_{j=1}^{k-1}v_i^j D_j I^1(p),\ldots,\sum_{j=1}^{k-1}v_i^j D_j I^k(p)\right)$$
So that means
$$I^*(dx^i)(p)(v_{i}) = \sum_{j=1}^{k-1}v_i^j D_j I^i(p) = \sum_{j=1}^{k-1} D_j I^i \;dx_j (p)(v_i) $$
And we have
$$I^*(dx^i) =\sum_{j=1}^{k-1} \frac{\partial I^i}{\partial x_j} \;dx_j $$
Now if $i = j$, then $I^i = \alpha$ so $I^*(dx_i) = 0$ and if $i\neq j$ then
either $i < j$ and so $I^*(dx^i) = dx^i$ or $i > j$ in which case $I^*(dx^i) = dx^{i-1}$.
Therefore if $i=j$
$$I^*(dx^1\wedge \cdots \wedge dx^{i-1}\wedge \widehat{dx^i} \wedge dx^{i+1} \wedge \cdots \wedge dx^{k})\\
=
dx^1\wedge \cdots \wedge dx^{i-1}\wedge \widehat{\,0\,} \wedge dx^{i+1-1} \wedge \cdots \wedge dx^{k-1}\\
=dx^1\wedge \cdots \wedge dx^{k-1}$$
If however $i\neq j$, then the $0$ occurs at a non-disappearing term, which makes the entire expression disappear, so altogether we have:
$$I^*(dx^1 \wedge \cdots \wedge \widehat{dx^i} \wedge \cdots \wedge dx^k) = \begin{cases}0 & i \neq j\\
dx^1\wedge \cdots \wedge dx^{k-1} & i= j\end{cases}$$
and so our original expression becomes:
$$\int_{[0,1]^{k-1}} I^* (f \; dx^1 \wedge \cdots \wedge \widehat{dx^i} \wedge \cdots \wedge dx^k) \\
= \begin{cases}0 & i \neq j \\
\int_{[0,1]^{k-1}} f(x_1,\ldots, x_{j-1},\alpha,x_j,\ldots,x_{k-1}) dx^1\cdots dx^{k-1} & i=j \end{cases}$$
Last thing to note is just that
$$\int_{[0,1]^{k-1}} f(x_1,\ldots, x_{j-1},\alpha,x_j,\ldots,x_{k-1}) dx^1\cdots dx^{k-1}\\
= \int_{[0,1]^{k-1}} f(x_1,\ldots, x_{j-1},\alpha,x_{j+1},\ldots,x_{k}) dx^1\cdots dx^{k}$$
with a change of variable for the last $k - j$ variables from $x^l \mapsto x^{l+1}$ and integrating along another singular 1-cube.