Great question!
You're absolutely right that we can think of a sequence of (not necessarily nested) sets as a function from $\mathbb{N}$ to $\textbf{Sets}$, and that - to the extent that we can think about $\mathbb{N}$ and $\textbf{Sets}$ as spaces - we might have a notion of derivative. However, it turns out that $\mathbb{N}$ is too bad for this to work nicely.
In order to make sense of the "derivative" of a function from $X$ to $Y$, we need $X$ and $Y$ to both carry a notion of distance - that is, we need $X$ and $Y$ to be metric spaces. On the face of it, this is then enough to define a difference quotient (EDIT: as lisyarus pointed out, this does not generalize the derivative, but rather the absolute value of the derivative: without a notion of direction, it is impossible to tell whether a function is "increasing" or "decreasing," etc.) as follows: $f'(a)$ is the limit, as $b$ approaches $a$, of the ratio $${d_Y(f(a), f(b))\over d_X(a, b)}$$ (where $d_X$ and $d_Y$ are the notions of distance on $X$ and $Y$, respectively). However, there is a huge problem with this: we're assuming that this limit, if it exists, is unique. In general, this won't be the case. For example, consider the natural metric on $\mathbb{N}$ ($d(m, n)=\vert m-n\vert$). This is full of "gaps," and these gaps prevent the derivative from making sense: given any function $f$ from $\mathbb{N}$ to $Y$, any $n\in\mathbb{N}$, and any real number $r$, we have $$\forall \epsilon>0\exists \delta>0\forall k[0<\vert k-n\vert<\delta\implies \vert{d_Y(f(a), f(b))\over \vert k-n\vert}- r\vert<\epsilon]$$ for stupid reasons: take $\delta<1$. So the statement "the difference quotient tends to $r$," as naively written, will be true for all $r$.
Basically, in order to have a useful notion of derivative of a function from metric space $X$ to metric space $Y$, we need $X$ to have no isolated points. If there are no isolated points in $X$, then the limit of the difference quotient (if it exists) is unique, so the theory of differentiation isn't totally broken. Of course, without further niceness assumptions on both $X$ and $Y$ it probably won't be good . . .
In case you're interested in crazy examples of this sort of thing, which are actually mathematically useful:
Most of the time you'll hear about calculus being generalized to manifolds; these are spaces that "look like" $\mathbb{R}$ in some sense, so the existence of such a generalization isn't really very surprising. For an example of calculus done on a wildly different space than $\mathbb{R}$, check out p-adic calculus: http://www2.math.ethz.ch/education/bachelor/seminars/hs2011/p-adic/report8.pdf