Let's first agree that the claim that the base ideal is the image of the evaluation map is true if and only if the scheme-theoretic support of the cokernel is precisely the base locus. This is essentially because we get a short exact sequence $$0 \rightarrow \text{im}(\text{eval}_{|D|}) \rightarrow \mathcal{O}_X \rightarrow \text{coker}(\text{eval}_{|D|}) \rightarrow 0$$ and so the ideal sheaf on the left cuts out the support of sheaf on the right (the latter is consequently the structure sheaf of that support).
Now we just need to establish that $p \in X$ is in $\text{Supp}(\text{coker}(\text{eval}_{|D|}))$ if and only if $s(p) = 0$ for every $s\in H^0(X,D)$. But of course, restricting to $p$, the evaluation map $H^0(X,D) \rightarrow \mathcal{O}_X(D)\vert_p$ (defined by $s \mapsto s(p)$) is surjective if and only if it is non-zero. i.e. if and only if $s(p) \not= 0$ for some $s$. Thus the points $p \in X$ where evaluation is not surjective (i.e. the support of the cokernel) is exactly the points $p$ where $s(p) = 0$ for all $s \in H^0(X,D)$.