In short, the mistake occurred because we need to look at absolute value.
The definition is: If $f(x)$ is $O(g(x))$, then for some $c,N\in\mathbb{R}$ we have $|f(x)|\le c|g(x)|$ for all $x\ge N$.
Now for our specific example, $f(n)=3n^2-100n+6$ and $g(n)=n^3$, so we have $f(1)=-91$ and $g(1)=1$. Now we need to check that $|f(1)|\le c|g(1)|$, which is not the case because $|f(1)|=91$ while $c|g(1)|=1\times 1=1$. Hence $N=c=1$ is not sufficient to prove that $f(n)$ is $O\left(n^3\right)$.
However, $3n^2-100n+6$ is certainly $O\left(n^3\right)$. If we choose say $c=3$ and $N=100$, then we have for every $n\ge N$
$$3n^3>3n^2>3n^2-100n+6>0$$
so we are done.