0

Intuitively, $\frac{\sum w_i a_i}{\sqrt{\sum w_i^2 b_i}}$ looks like a ratio between a $L_1$ norm and $L_2$ norm.

To provide motivation, we consider i.i.d. $(x_i, y_i)$ and $\text{Var}[x_i] = \sigma_{i}^2, \text{Var}[x_i] = \tau_{i}^2, \text{Cov}[x_i, y_i] = r_{i}$. We wish to find the linear combination of $\sum w_i x_i$ such that $\text{Cor}[\sum w_i x_i, \sum y_i]$ is maximized.

$$ \text{Cor}\left[\sum w_i x_i, \sum y_i \right] \propto \frac{\sum w_i \text{Cov}[x_i, y_i]}{\sqrt{\sum w_i^2 \text{Var}[x_i]}} = \frac{\sum w_i r_i}{\sqrt{\sum w_i^2 \sigma_i^2}} $$

david
  • 73

1 Answers1

0

Define two vectors:

$\mathbf{x} = (w_1 \sqrt{b_1}, w_2 \sqrt{b_2}, \ldots, w_n \sqrt{b_n})$, $\mathbf{y} = (\frac{a_1}{\sqrt{b_1}}, \frac{a_2}{\sqrt{b_2}}, \ldots, \frac{a_n}{\sqrt{b_n}})$

$\mathbf{x} \cdot \mathbf{y} = \sum w_i a_i$

and

$\mathbf{x} \cdot \mathbf{x} = \sum w_i^2 b_i, $ $\mathbf{y} \cdot \mathbf{y} = \sum \frac{a_i^2}{b_i}$

Applying the Cauchy-Schwarz inequality:

$(\sum w_i a_i)^2 \leq (\sum w_i^2 b_i)(\sum \frac{a_i^2}{b_i})$

Therefore, the maximum value of the original expression is:

$\frac{\sum w_i a_i}{\sqrt{\sum w_i^2 b_i}} \leq \sqrt{\sum \frac{a_i^2}{b_i}}$

The equality holds when $\mathbf{x}$ and $\mathbf{y}$ are linearly dependent, which means:

$w_i \sqrt{b_i} = c \cdot \frac{a_i}{\sqrt{b_i}}$

for some constant $c$.

For our motivating example, $$ \text{Cor}\left[\sum w_i x_i, \sum y_i \right] \propto \frac{\sum w_i \text{Cov}[x_i, y_i]}{\sqrt{\sum w_i^2 \text{Var}[x_i]}} = \frac{\sum w_i r_i}{\sqrt{\sum w_i^2 \sigma_i^2}} $$

$w_i$ should satisfy $w_i \propto \frac{r_i}{\sigma_i^2}$, which is the regression slope of regressing $y_i \sim x_i$.

david
  • 73