I have a constrained nonlinear least squares minimisation problem set up (using Python scipy's SLSQP solver) that looks something like:
$$\min_w \frac{1}{N}\sum_i (P_i-O_i)^2,$$ where $P=\sum_j w_j p_j, \ \ \ \ $ $p_j$ are prediction probabilities and $O$ is an indicator function 0 or 1 (True or False). The constraint is that $\sum_j w_j=1$. This basically is the Brier score, minimised over $j$ different predictors, averaged over all events $N$.
What I would like is to add some temporal decay on the residuals. The reason is that I want to discard residuals (i.e. predictions) that are too old, using some kind of smooth function. So assuming that the events $i \in 1...N$ are ordered by time, I was thinking something like: $$ \min \frac{1}{N}\sum_i \exp(i)(P_i-O_i)^2. $$
For some reason I am not sure this is the best way to incorporate a (say, exponential) decay into the model. Can you suggest something else/better? I'd rather put the decay into the formula, than individual weights on the residuals $r=(P_i-O_i)$ on the Python solver.
