There are a few different definitions of "subbasis," and without knowing exactly which version your text uses, the approach may vary.
The most common I've seen are as follows:
- Given a set $X,$ a topology $\mathcal T$ on $X,$ and a collection $\mathcal S$ of subsets of $X,$ we say that $\mathcal S$ is a subbasis for $\mathcal T$ if $\mathcal T$ is the set of unions of finite intersections of elements of $\mathcal S.$ (Here, the empty intersection $\bigcap\emptyset$ is taken to be $X.$)
- Given a set $X,$ a topology $\mathcal T$ on $X,$ and a collection $\mathcal S$ of subsets of $X,$ we say that $\mathcal S$ is a subbasis for $\mathcal T$ if the collection of finite intersections of elements of $\mathcal S$ comprise a basis for $\mathcal T.$
- Given a set $X,$ a topology $\mathcal T$ on $X,$ and a collection $\mathcal S$ of subsets of $X,$ we say that $\mathcal S$ is a subbasis for $\mathcal T$ if $\mathcal T$ is the coarsest topology on $X$ that contains $\mathcal S$--that is, if $\mathcal S\subseteq\mathcal T,$ and given any topology $\mathcal T'$ on $X$ with $\mathcal S\subseteq\mathcal T'$ we have $\mathcal T\subseteq\mathcal T'.$
The three definitions above are equivalent. Another common one is:
- Given a set $X,$ a topology $\mathcal T$ on $X,$ and a collection $\mathcal S$ of subsets of $X,$ we say that $\mathcal S$ is a subbasis for $\mathcal T$ if $X=\bigcup\mathcal S$ and $\mathcal T$ is the coarsest topology on $X$ that contains $\mathcal S.$
If $X$ has at least two points and we are considering topologies in which one-point subsets of $X$ are closed, then this definition is equivalent to the others, but need not be, in general. Fortunately, in an order topology, one-point subsets will be closed, so the definition is largely immaterial. Under the first two definitions, it is readily shown that any element of $\mathcal S$ will be an element of $\mathcal T,$ and in the other two definitions this is already explicit.
In particular, if $b$ is the largest element of $X,$ then we'll have $(a,b]=\{x\in X:x>a\},$ which is then an element of the given subbasis, so an open set.
I note that there is an error in the Case 2 proof you site from your book. It should instead consider the set $A=\{x\in X:x>b\}.$ Then $A$ is non-empty by the Case 2 assumption, and so has the least element $c$ used in the rest of that proof.
As for justifying why $(a,c)\in\mathscr T,$ that works perfectly! It follows a bit more immediately from the first and second definitions of subbasis mentioned above, but only slightly.