5

It is well-known that Hamilton's equations $$\dot{x}^\mu=\Omega^{\mu\nu} \frac{\partial H(x)}{\partial x^\mu}\tag{1}$$ where $\Omega$ is the symplectic form and $x^\mu=(q,p)$ follow from a Lagrangian, namely $$L(x)=\dot{q}p-H(x).\tag{2}$$

Gradient descent is given by the equation $$\dot{x}^\mu=-G^{\mu\nu}\frac{\partial f(x)}{\partial x^\mu},\tag{3}$$ where $G$ is a Riemannian metric and the flow decreases $f(x)$. My intuition is that there is no way to get the gradient descent equations from a Lagrangian action, but I'm curious if this is true and why not?

Qmechanic
  • 201,751
LFH
  • 730
  • I am not sure to understand. Are you asking if, given $H$, there is a Riemannian metric $G$ such that your second equation has the same solutions (motions) as your first equations? The answer is positive only if $H$ is constant and the solutions of Hamilton equations are trivial. – Valter Moretti Dec 08 '19 at 08:00
  • My question is if there is a Lagrangian, such that its equations of motion correspond to gradient descent (second equation). Of course, the Hamiltonian could be completely different - the only requirement is that the equations of motion describe gradient descent of a function. I now called this function $f$. – LFH Dec 08 '19 at 10:44

2 Answers2

2

  1. Let us rewrite OP's eq. (3) as $$ \dot{x}^j~=~-g^{jk}(x)\frac{\partial V}{\partial x^k} \qquad \Leftrightarrow \qquad g_{jk}(x)\dot{x}^k~=~-\frac{\partial V}{\partial x^j}. \tag{3}$$ We note that this dynamics has no time-reversal symmetry.

  2. Moreover, eq. (3) implies that $$ V(x_i)-V(x_f) ~=~ \int_{t_i}^{t_f} \! \mathrm{d}t ~\dot{x}^jg_{jk}(x)\dot{x}^k~\geq~0 . $$ We conclude that this dynamics cannot have closed orbits, and that it is dissipative in nature.

  3. If we consider a sufficiently small neighborhood, we can use Riemann normal coordinates, and assume that the metric components $g_{jk}$ are constant. We can even assume that the metric components $g_{jk}$ are diagonal. By furthermore scaling the $x^j$-coordinates, we may assume that the metric components $g_{jk}=\delta_{jk}$. More generally, this is known as Aristotelian mechanics, which has no conventional stationary action principle. See also this Phys.SE post.

Qmechanic
  • 201,751
0

Ok, I think I figured out the argument: For gradient descent, all solutions around a global minimum will converge to this minimum. As (time-independent) Lagrangian dynamics is equivalent to Hamiltonian dynamics, it is clear the energy (value of Hamiltonian) is preserved under time evolution. By continuity, the value of the Hamiltonian along a solution should be the same as at the minimum. However, this implies that the Hamiltonian must be constant in some neighborhood as all solutions should converge to the same minimum. Consequently, for a general function $f$, there is no Lagrangian action.

LFH
  • 730