2

Suppose you have a multi-variable functional given as

$$ \int_{t_0}^{t_1} L(t,x,x', x'' ..., y, y', y'', ... ) \ dt $$

That you wish to to optimize [i.e find an $x$ and $y$ that maximize] but subject to a constraint

$$ \Omega(t,x,x', x'', ... y,y', y'' ... ) = 0 $$

What would be a general framework for approaching a problem like this?

Attempting to Generalize from Simpler Cases:

The finite-dimensional analog of this is to try to optimize a multivariable function

$$ F(x,y) $$

subject to a constraint $$G(x,y)=0$$. The standard approach is to use lagrange multipliers i.e. finding $x,y,\Lambda \in \mathbb{R} $ such that

$$ \nabla [ F(x,y)] = \Lambda \nabla [ G(x,y) ] $$ $$ G(x,y) = 0$$

In doing so, essentially the ingredients we need are

  1. A notion of "functional gradient" which describes how $x,y$ ought to be perturbed to increase the value of our functional

  2. A notion of a "functional tangent" which describes locally how a functional looks like

  3. A notion of projection of the "functional gradient" to the "functional tangent"

Once you have these in place then a version of lagrange multipliers should be possible in the infinite dimensional case.

A dumb approach:

In theory lagrange multipliers are never "needed" they are merely elegant convenient and efficient.

Given The problem, minimize/maximize

$$ f(x_0, x_1, x_2 ... x_n) $$

subject to

$$ g(x_0, x_1, x_2 .. x_n) = 0 $$

We can observe there will always be a collection of functions $H_k$ such that if

$$ g(x_0, x_1, x_2 .. x_n) = 0 $$

then

$$ x_0 = H_k(x_1 .. x_n)$$

And therefore the problem of minimizing/maximizing $f(x_0, ... x_n)$ subject to the constraint can be restated as minimizing/maximizing $f(H(x_1 .. x_n) x_1 , ... x_n)$ that is completely unconstrained. It's obvious how to generalize this to our infinite dimensional case, and that works!

Unfortunately a more elegant "lagrangian" approach still eludes me.

More notes:

It's been some time since I posed this problem. I don't yet have a rigorous proof but I am fairly confident just introducing lagrange multiplier $\lambda_i$ for each constraint and functionally differentiating each one to create a system of couple differential equations is the correct approach here. It seems almost all multivariable techniques generalize in the obvious way to the infinite dimensional case so far (except the chain rule), so i would be surprised if this was any different.

Sidharth Ghoshal
  • 16,771
  • 1
    Just satisfying your constraint (with no concern about objective function maximization) requires you to solve a general high order nonlinear ODE. How do you propose to do that!? – Michael Sep 22 '19 at 00:43
  • When these things can be solved, they often do not require much more than basic Lagrange multiplier theory in $\mathbb{R}^2$. On the other hand, with no structure given, you cannot give a solution and you are likely left to numerical methods, in which case you can discretize and not worry about infinite dimensions. This closely relates to my first comment that you seem to have ignored. – Michael Sep 22 '19 at 01:04
  • I’m just looking for an abstract approach with intuition. Realistically even Lagrange multipliers is impractical when you have non linear functions of many variables, but conceptually it’s intuitive why it works and serves as a mental model of how, given unlimited resources, to solve any type of constrained functional optimization problem. I’m hoping to find something similar here in the infinite dimensional world. – Sidharth Ghoshal Sep 22 '19 at 01:06
  • Here is an example of using Lagrange multipliers for finding a closed-form solution for an optimal function $q(x)$ for $x$ defined over an abstract set $E$: https://math.stackexchange.com/questions/3313495/minimize-q-mapsto-int-fracpf2q-rm-d-lambda-subject-to-int-q-rm/3323806#3323806 – Michael Sep 23 '19 at 18:00
  • In your case Lagrange multipliers might be better suited for this problem: Define $\mathcal{F}$ as the set of all twice-continuously differentiable functions $f:[t_0, t_1]\rightarrow\mathbb{R}$ that satisfy a desired ODE. For each $f \in \mathcal{F}$, define $L_1(f) = \int_{t_0}^{t_1} h_1(f(t), f’(t), f’’(t))dt$ and define $L_2(f) = \int_{t_0}^{t_1} h_2(f(t), f’(t), f’’(t))dt$ (where $h_1, h_2$ are given functions). Fix $c \in \mathbb{R}$ and consider the problem: \begin{align} \mbox{Minimize:} & L_1(f)\ \mbox{Subject to:} & L_2(f)=c\ & f \in \mathcal{F} \end{align} – Michael Sep 23 '19 at 18:18
  • There are direct relationships to the "unconstrained problem" of minimizing $L_1(f) + \lambda L_2(f)$ over $f \in \mathcal{F}$, where $\lambda \in \mathbb{R}$ is a “Lagrange multiplier.” – Michael Sep 23 '19 at 18:22

1 Answers1

1

I would start with the wiki page on Lagrange multipliers in Banach spaces and go from there. This theory has definitely been thoroughly explored, as it has lots of applications to Calculus of Variations.

Jake Mirra
  • 3,198
  • i'm looking at the link here: https://en.wikipedia.org/wiki/Lagrange_multipliers_on_Banach_spaces, unfortunately im not familiar with concepts like "frechet derivative" or "pullbacks" is there a book or resource you recommend to read that makes this accessible? – Sidharth Ghoshal Sep 22 '19 at 00:44
  • I learned from lectures and "here-and-there" during my PhD. I was really hoping someone else would chime in here because I don't think I have the best references for the subject. But I do know for a fact that the book "Calculus of Variations" by Gelfond and Fomin has an explanation and examples of the "functional lagrange multipliers" technique you're looking for---and it's not a super theory-heavy book. I'd actually say it's written more for an engineer type, which might be ideal since you don't have a lot of background in Functional Analysis. – Jake Mirra Sep 29 '19 at 19:13