Let $f:[a,b] \to \mathbb{R}$ be some bounded function. Let $\mathcal{P} = \{a = x_0 < x_1 < ... < x_N = b\}$ be any partitioning of $[a,b]$. We define the (left) Riemann sum of $f$ corresponding to $\mathcal{P}$ by $\mathcal{R}_{\mathcal{L}}(f,\mathcal{P}) = \sum_{j=1}^N f(x_{j-1})(x_j - x_{j-1})$.
Define the mesh of $\mathcal{P}$ as $|\mathcal{P}| := \max \{x_j - x_{j-1}: 1 \le j \le N \}$. We say that $f$ is Riemann integrable if $\lim_{|\mathcal{P}| \to 0} \mathcal{R}_{\mathcal{L}}(f, \mathcal{P}) $ exists, in which case we denote this limit as $\int_a^b f(x) dx$.
Suppose now that we have a function $f:(a,b] \to \mathbb{R}$. For each $c >a$, we can ask whether $\int_c^b f(x) dx$ exists. If it does, and if $\lim_{c \to a^+} \int_c^b f(x) dx$ exists, we say that this value is an improper Riemann integral.
We similarly define $\int_a^{\infty} f(x) dx := \lim_{n \to \infty} \int_a^n f(x) dx$ and $\int_{-\infty}^b f(x) dx := \lim_{n \to -\infty} \int_n^b f(x) dx$.
In any case, the improperness only refers to the definition of the integral we're using, where we take a limit of well defined integrals to assign a new value to an integral that is not defined under our ordinary definition.