First question:
The terms linear system and linear series are completely interchangeable, it's just a matter of taste. Here's the definition of a linear system of divisors:
Def: A divisor $D$ is linearly equivalent to $D'$ if there exist a globally defined rational function $f:C\to k$ such that $D+(f) = D'$.
Def: A divisor $D$ is effective if the order of every point is non-negative.
Def: Given a divisor $D$ on a curve $C$, the complete linear system (or complete linear series) $|D|$ associated to $D$ is the set of all effective divisors on $C$ which are linearly equivalent to $D$.
Def: A linear system (or linear series) is a linear subspace of a complete linear system.
Second question:
Recall that the Riemann-Roch theorem for smooth curves states that
$$ h^0(D) - h^0(K-D) = \deg(D) - g + 1, $$
where:
$g$ is the genus of the curve $C$ which, by the genus-degree formula for smooth plane curves, is given by
$$ g = \frac{(n-2)(n-1)}{2} $$
$h^0(D)$ denotes the dimension of the linear series $|D|$ as a vector space over the ground field $k$
$K$ is any canonical divisor of $C$ and $h^0(K-D)$ denotes the dimension of the linear series $|K-D|$ as a vector space over the ground field $k$
Further, recall that as soon as $\deg(E)<0$ we have $h^0(E) = 0$, i.e. the linear series $|E|$ consists of $E$ only.
Now, since the degree of a canonical divisor $K$ is given by (to see this just plug $D=0$ in the Riemann-Roch formula above)
$$ \deg(K) = 2g-2 = n\cdot(n-3), $$
we deduce that, if $D$ is the divisor of degree $n\cdot d$ consisting of the points of intersection between $C$ and another plane curve of degree $d$, we have
$$ d > n-3 \implies \deg(D)>\deg(K) \implies h^0(K-D) = 0. $$
Therefore in the case $d > n-3$ the dimension of the linear series $|D|$ can be easily computed using the Riemann-Roch formula: in this case indeed we have
$$ h^0(D) = n\cdot d - g + 1 = \frac{n\cdot d - (n-2)(n-1) + 2}{2} = \frac{n\cdot (2d -n+ 3)}{2}. $$
On the other hand, if $d \leq n-3$ the dimension of $|D|$ is harder to compute, because of the tricky term $h^0(K-D)$ appearing in the Riemann-Roch formula.