Consider the expression
$$ J(y) := \int_a^b F(x,y,y_x) dx. $$
Here, $y$ is a function: $y:[a,b]\rightarrow \mathbb{R}$.
Note: $y_x$ is a common notation for the partial derivative. Since $y$ depends only on one variable, $y_x \equiv y'$. The above expression abuses notation (in a very common way), and a better version is
$$ J(y) := \int_a^b F(x,y(x),y_x(x)) dx. $$
Note that $F$ is not required to have vanishing derivatives of any order; it is just required to be sufficiently smooth. The function $F : \mathbb{R}^3\rightarrow \mathbb{R}$, and is to be evaluated at $x$, $y(x)$, and $y_x(x)$. The integrand is a well-defined function whose integral is guaranteed to exist.
Thus, $J$ takes a function $y : [a,b] \rightarrow \mathbb{R}$ and computes a number $J(y)$ by the right hand side. Physicists often call such a "function of a function" by the name "functional". When analyzed mathematically, $y$ is often described as an element in an abstract vector space of infinite dimension, call it $X$. Thus, think of $J$ as a map that takes a vector $y$ in $X$(visualize a finite dimension if you like) and produces a number.
The regularity assumtion of $F$ and $y$ lets you compute the change in $J(y)$ when $y$ changes by a "small amount", meaning that a small function is added to $y$. Typically, physicists think intuitively here, but analytst turn to the space $X$ which comes equipped with a rigorous way to measure smallness, for example a norm.
In physics notation, $y(x) \rightarrow y(x) + \epsilon \eta(x)$ for such a small change, and then compute the Taylor expansion:
$$ J(y + \epsilon \eta) \approx \int F(x,y+\epsilon\eta,y_x + \epsilon \eta_x) dx = \int F(x,y,y_x) dx + \int F_2(x,y,y_x)\epsilon \eta(x)dx + \int F_3(x,y,y_x)\epsilon y_x dx, $$
where $J_i$ is the partial derivative with respect to argument number $i$ of $F$. Physicists work from this expression and derives the Euler--Lagrange equations.