Recall that in the product toppology, sets of the form
$$\tag1U_{i,a}:=\{\,(x_\iota)_{\iota\in I}\colon x_i=a\,\}$$
are open, as are their finite intersections
$$\tag2\bigcap_{r=1}^m U_{i_r,a_r}$$
and ultimately the arbitrary (though in this example: wlog. finite) unions of sets of form $(2)$. In order to be dense, our desired subgroup must intersect every non-empty open set, or equivalently must intersect every set of form $(2)$.
If $H$ is of index $p^n$ in $\Bbb F_p^I$, then clearly $\Bbb F_p^I/H\cong \Bbb F_p^n$, which is a nice vector space, and $H$ can be described as kernal of a homomoprhism to a finite-dimensional vector space. Let's start with the basic case $n=1$. What kind of onto homomorphisms $\Bbb F_p^I\to \Bbb F_p$ can we think of? The simplest ones are projections to some coordinate, $(x_\iota)_{\iota\in I}\mapsto x_{\iota_0}$. That would make $ H=\{\,(x_\iota)\mid x_{\iota_0}=0\,\}$, so $H\cap U_{\iota_0;1}=\emptyset$, and thus $H$ is unfortunately not dense.
We need a different kind of homomorphism!
On the powerset $\mathcal P(I)$, define an equivalence relation $A\sim B\iff |A\mathop\Delta B|<\infty$. Then every equivalence class has only countably many members, hence $J:=\mathcal P(I)/{\sim}$ is as uncountable as $\mathcal P(I)$ is.
For each $j\in J$ except the one with $\emptyset\in j$, we choose a representative $A_j\in j$ and define $e_j=(e_{j;\iota})_{\iota\in I}$ as
$$e_{j;\iota}=\begin{cases}1&\iota\in A_j\\0&\iota\notin A_j\end{cases} $$
Additionally, we have for each $i\in I$ a vector $f_i$ given by
$$f_{i;\iota}=\begin{cases}1&\iota=i\\0&\iota\ne i\end{cases} $$
Then these $e_j$, $f_i$ are linearly independent: If
$$\tag3\sum_{j\in J} a_je_{j}+\sum_{i\in I}b_if_i=0,$$
where only finitely many coefficients are non-zero,
then for each $j$ with $a_j\ne 0$, the summand $a_je_j$ is non-zero in infinitely many components where none of the other summands of the first sum is non-zero. As the second sum is non-zero only in finitely many components, at least one component remains where the left hand side of $(3)$ is non-zero, contradiction. Therefore $a_j=0$ for all $j\in J$. But for the second sum it is directly clear that $b_i=0$ for all $i\in I$, as was to be shown.
Thus (using the Axiom of Choice a second time) we can extend the collection of all $e_j$ and all $f_i$ to a basis $\{e_j\}_{j\in J}\cup \{f_i\}_{i\in I}\cup \{h_k\}_{k\in K}$ of $\Bbb F_p^I$.
Now pick distinct $j_1,\ldots, j_n\in J$ and let $H$ be the kernel of
$\Bbb F_p^I\to \Bbb F_p^n$,
$$\sum_{j\in J}a_je_j+\sum_{i\in I}b_if_i+\sum_{k\in K}c_kh_k\mapsto (a_{j_1},\ldots, a_{j_n}). $$
Then $H$ is of index $p^n$ and intersects every set of form $(2)$ because we can pick any $x=\sum a_je_i+\sum b_if_i+\sum c_kh_k\in H$ and adjust the coefficient $b_i$ to achieve any desired value in the $i$th component, and repeat this for the indices $i_r$, $1\le r\le m$ given in $(2)$.