What is wrong in the following? \begin{equation*} g_{\mu\nu}(x)g^{\mu\nu}(x)=D \\ \frac{\delta}{\delta g_{\alpha\beta}(x)}D=0 \\ \frac{\delta}{\delta g_{\alpha\beta}(x)}(g_{\mu\nu}g^{\mu\nu})=2g^{\alpha\beta}(x)=0 \end{equation*} The above equations are clearly inconsistent.
I am just trying to figure out what is the correct way of defining the correct functional derivative of $g_{\mu\nu}g^{\mu\nu}$ with respect to the components of $g$ itself.
Your question is posed rather weirdly...
And I am quite sure you cannot really differentiate it like that. Instead of attempting to calculate a Fréchet-derivative directly in a wrong way, try calculating a Gateux-derivative along an arbitrary curve in the space of possible metrics, eg. take $$ \left.\frac{d}{d\varepsilon}(g_{\mu\nu}(\varepsilon)g^{\mu\nu}(\varepsilon))\right|_{\varepsilon=0}, $$ where $g_{\mu\nu}(0)=g_{\mu\nu}$, with a bit of abuse of notation.
Edit: You seem to be rather confused about functional derivation.
First of all, the map $g_{\mu\nu}\mapsto g_{\mu\nu}g^{\mu\nu}$ is not really a functional in the sense that these type of maps lead into $C^\infty(M)$, not $\mathbb{R}$. However in this case it maps into a constant function, which can be viewed as an element of $\mathbb{R}$, thus it is a functional. It is also obvious that since $g^{\mu\nu}$ is not independent from $g_{\mu\nu}$, such expression for any sort of metric will always result in the same number, thus, the derivative must be zero, since the "functional" is constant.
Secondly, the way you tried to use chain rule here is just wrong, which is why you got nonzero result. In general, you cannot calculate a functional derivative "directly".
Most of the time in physics, these functionals will take the form $$ F[\phi]=\int L(\phi,\partial\phi,...,\partial^k\phi)\ \Omega ,$$ where $\Omega$ is some volume element. In this case, the functional derivative always exists (provided $L$ is a well behaved function), and a directional derivative at $\phi$ in the direction of some arbitrary function $\delta\phi$ (which we usually define as a tangent vector to some curve in the function space, eg. $\delta\phi=d\phi/d\varepsilon$) will always take the form $$ \frac{dF}{d\varepsilon}=\int\frac{\delta F}{\delta\phi}\delta\phi\ \Omega .$$ In this case we call $\delta F/\delta\phi$ the functional derivative of $F$, and obviously the above expression is linear in $\delta\phi$.
If the functional is NOT in this integral form, then we cannot really a define a functional derivative like that, rather, if the functional is differentiable, then there must exist some linear map from the space of functions to the reals, for which the directional derivative in the direction of an arbitrary function always equals the action of that linear map on the function.
Therefore, what you are really interested in is actually the expression $$ \left.\frac{d}{d\varepsilon}(g_{\mu\nu}(\varepsilon)g^{\mu\nu}(\varepsilon))\right|_{\varepsilon=0}, $$ where $g_{\mu\nu}(\varepsilon)$ is a family of metrics indexed by a real parameter in a smooth way, and the metric at the 0 parameter is the metric you want to perturb.
Can you take it from here?
