It is difficult to answer your question because you do not give much context. However, I am sure that for a covering map $p : E \to B$ an elementary open neigborhood of $b \in B$ is that what is usually denoted as an evenly covered neigborhood. This an open neigborhood $U$ of $b$ such that $p^{-1}(U) = \bigcup_{\alpha \in A} V_\alpha$ with pairwise disjoint open $V_\alpha \subset E$, all of which are mapped by $p$ homeomorphically onto $U$. That is, all restrictions $p_\alpha : V_\alpha \stackrel{p}{\to} U$ are homeomorphisms. Certainly each of the homeomorphims $p_\alpha^{-1} : U \to V_\alpha$ is what the author understands as a local inverse for $p$.
This answers Q2 : An arbitrary open neigborhood need not be elementary open neigborhood. As an example take the covering map $p : \mathbb R \to S^1, p(t) = e^{2\pi it}$. Each point of $S^1$ has $S^1$ itself as an open neigborhood, but $S^1$ is certainly not an elementary open neigborhood.
Concerning Q1 : "We want to find an elementary neighborhood around $y_2$ together with a local inverse for $F$". This is just what the definition of a covering map requires.
Remark:
It seems that the Theorem needs as an aditional assumption that $X$ is locally path connected (or at least that $X$ is locally connected). Perhaps this is a general assumption made by the author for all covering maps?
See Exercise 1.3.16 in Hatcher to understand where the existence of arbitrarily small connected neigborhoods is needed.
