0

I have come across the following argument concerning the Bayes error:

Let $r(x)=\mathbb{E}[Y|X=x]$ be the regression of $Y$ on $X$. Then one can show that the Bayes error is

$L^* = \mathbb{E}[\min(r(X),1-r(X))]$.

Suppose now that $X$ has a density $f$. Denote $p=\mathbb{P}[Y=1]$. Then

$L^* = \int \min (r(x),1-r(x)) \, f(x) \, dx = \int \min ((1-p)f_0(x),pf_1(x)) \, dx$,

where $f_0,f_1$ are the class-conditional densities. Now comes the following claim:

If $f_0$ and $f_1$ are non-overlapping, i.e. $\int f_0(x) f_1(x) \, dx = 0$, then obviously $L^*=0$.

I cannot see at a glance how the property of being non-overlapping is related to orthogonality in $L_2$. Neither is it obvious to me how the expression for $L^*$ can be bounded from above by $\int f_0(x) f_1(x) \, dx = 0$. Is some elementary inequality for $\min$ involved here? What am I missing?

HAL9000SE
  • 31
  • 1
    Non-overlapping means, that whenever $f_0(x)\neq 0$ then $f_1(x)=0$ and vice versa. If $f_0$ and $f_1$ are non-overlapping, then $(1-p)f_0$ and $pf_1$ are non-overlapping too. Moreover it follows directly from the definition, that $\min(g,h)=0$ for non-overlapping non-negative functions $g,h$. – Hagen Knaf Mar 22 '17 at 13:21
  • Ah, the meaning of non-overlapping is really easier than I thought! Alright, but from $\int f_0f_1 , dx =0$ it follows that $f_0(x)f_1(x)=0$ for almost all $x$, that is, it may well happen that $f_0(x)f_1(x) \neq 0$ at some points. Is it really an equivalence then? – HAL9000SE Mar 22 '17 at 14:17
  • 1
    OK, I see there is a subtle point in your post: the 7th line can also be read as a definition of "non-overlapping". Then indeed the set ${x\in\mathbb{R} : f_0(x)\neq f_1(x)}$ is a set of measure $0$. But this set is exactly the set of non-zero function values of $\min(pf_0,(1-p)f_1)$, hence the integral over that function must be zero. – Hagen Knaf Mar 22 '17 at 20:40

0 Answers0