This is the standard definition, and one of the inequalities has to be strict, as I will show: The intuition goes as following: Saying that a point $x$ is a limit point of a set $A$ is the same as saying that "however close I get to $x$, there is always a point of $A$ nearby". So basically, for every small interval around $x$, I want there to be a point of $A$ in it. Using $\epsilon$ notation, for every small positive $\epsilon$, I want there to be a point of $A$ in the interval $(x-\epsilon,x+\epsilon)$. But this is the same as saying that for every $\epsilon>0$, there is a point $y\in A$ that is closer to $x$ than $x-\epsilon$ or $x+\epsilon$; that is,
$$x-\epsilon<y<x+\epsilon.$$
By subtracting $x$, we get that $-\epsilon<y-x<\epsilon$, and this is the definition of $|y-x|<\epsilon$.
It is easy to see that you can tweak this a bit and say that we want a point of $A$ in $[x-\epsilon,x+\epsilon]$ for all $\epsilon$, and so we want all $y\in A$ such that $0<|y-x|\leq \epsilon$.
We can't have $|y-x|=0$, however, because that just says that $x=y$, and we want points of $A$ that are close to $x$ but not equal to $x$.