This means to find a 3rd line segment $EF$ (i.e. a unit segment) such that
$$|AB| / |EF| = n$$
$$|CD| / |EF| = m$$
where $m$ and $n$ are integers (or say rational numbers)
Here by $|XY|$ we denote the length of the line segment $|XY|$
The Greeks "thought" this was possible but it turned out it's not.
E.g. if $AB$ is the side of a square and $CD$ is one of its diagonals,
you cannot find such a segment $EF$.
The discovery in question here is the one stated in the book:
there's no rational number whose square equals $2$.
Here is the quote from the book:
Prior to the preceding discovery, it was an assumed and commonly used
fact that, given two line segments AB and CD, it would always be
possible to find a third line segment whose length divides evenly into
the first two. In modern terminology, this is equivalent to asserting
that the length of CD is a rational multiple of the length of AB.
Looking at the diagonal of a unit square (Fig.
1.1), it now followed (using the Pythagorean Theorem) that this was not always the case. Because the Pythagoreans implicitly interpreted
number to mean rational number, they were forced to accept that
number was a strictly weaker notion than length.
Rather than abandoning arithmetic in favor of geometry (as the Greeks
seem to have done), our resolution to this limitation is to strengthen
the concept of number by moving from the rational numbers to a larger
number system.
That larger number system implied here is the system of
the real numbers which serves as the base for real analysis.
In plain words the idea of what the book text says is this. Whatever unit segment you pick, you can always find segments whose lengths cannot be expressed as a rational number times the unit segment.
Such segments are those whose lengths happen to be an irrational number. E.g. if the unit segment is $1$ meter, then a segment whose length you cannot express is the diagonal of the unit square. Another segment which you cannot express is the diagonal of a rectangle with sides $1$ meter and $2$ meters.
So basically Greeks thought that segment lengths are a proper super set of numbers (i.e. that there're segments whose lengths you cannot express as numbers). That's because they knew only the rationals, not the reals. This is a rephrased version of the book's text.
