Here are the correct statements:
For $n\ne 4$, any compact $n$-dimensional topological manifold admits only finitely many smooth structures up to diffeomorphism.
There are compact 4-dimensional manifolds which admit (countably) infinitely many pairwise non-diffeomorphic smooth structures. An example is given by the K3-surface. See the first Corollary in section 12.4, p. 545, in
Alexandru Scorpan, The wild world of 4-manifolds, Providence, RI: American Mathematical Society (AMS) (ISBN 0-8218-3749-4/hbk). xv, 609 p. (2005). ZBL1075.57001.
Note that this theorem about K3 surfaces appears on page 545 of this nicely-written book. Depending on your background (assuming, say, that you are familiar with algebraic topology, classical differential topology, differential geometry), it will take you a year or two of reading this book to get there. (Most likely, you would need help from an advisor.)
- It is a conjecture/open problem: If $M$ is a 4-dimensional manifold which admits one smooth structure, then it admits infinitely many pairwise non-diffeomorphic smooth structures. The most famous open case is that of $S^4$: It is unknown if the 4-dimensional sphere admits even one exotic smooth structure.
The proof of 1 is even harder than the proof of 2. See my answer here. Unless you have an advisor in the relevant subfield of topology (there are very few, one is Shmuel Weinberger, at U of Chicago), you will never get through the book of Kirby and Siebenmann to the point of understanding the proof. Sorry for being so negative here, but this is just the reality. (I think, even Terry Tao will have hard time.)
If you want to know "why dimension 4 is so different," the rough answer is "because the Whitney trick fails in dimension 4 in the smooth category." Read Chapter I of the book by Scorpan to understand what this sentence even means. (Note that Whitney trick also fails in dimension 3, but every 3-dimensional manifold admits a unique smooth structure. Also, Whitney trick fails for non-simply connected manifolds of dimension $\ge 5$. So, the above answer is only a very rough approximation to the real one.)
On every compact manifold there are at most countably many smooth structures (up to diffeomorphism), see again my answer here. There is exactly one smooth structure on manifolds homeomorphic to ${\mathbb R}^n$, $n\ne 4$ (Kirby and Sibenmann), and there are uncountably many non-diffeomorphic smooth structures on manifolds homeomorphic to ${\mathbb R}^4$ (Gompf).