How to define $p$ and $q$ in Hamiltonian system?

Question

In Lagrangian mechanics, once we define $q$ which is about the position, then we automatically get $\dot q$ such that the data $(q,\dot q)$ uniquely determines the state of the system.

But in Hamiltonian mechanics, $p$ and $q$ are somehow independent. From my understanding, even if $(A,B)$ determines the state of a system, they may not be able to serve as generalized coordinates and momenta. My question is if we want to use Hamiltonian mechanics, how do I find the 'correct' selection of $p$ and $q$?

I know we can first write down Lagrangian and then do Legendre transform. But shouldn't Hamiltonian be an independent theory? It seems in some physics theories, for example quantum mechanics, people directly write down $(p,q)$ and work on Hamiltonian system without using Lagrangian. There should be a way to know which pair is appropriate candidate for $(p,q)$.

Answer 1

In addition to the previous nicely written answers, let me add a review, which includes both a mathematical and a physical perspective. Let us start with Lagrangian mechanics, then develop Hamiltonian mechanics independently and finally relate the two via the fiber derivative.

A Lagrangian system is a tuple $(Q,M,L)$, where:

1. $Q$ is an $n$-dimensional smooth manifold, called the configuration space;
2. $M=TQ$ is the tangent bundle of $Q$, a smooth $2n$-dimensional manifold, called the velocity phase space;
3. $L:TQ \to \mathbb{R}$ is a smooth function, called the Lagrangian of the Lagrangian system $(Q,M,L)$.

Note that in order for a Lagrangian system to be uniquely specified, you have to specify both the configuration space and the Lagrangian. In physics, we usually take specific Lagrangians, but note that this is a choice! We usually consider Lagrangians of the form kinetic energy- potential energy, which mathematically can be explicitly written as follows: \begin{equation} L:TQ \to \mathbb{R}, \; \; L(v)=m\frac{1}{2} g(v,v) - V(\pi(v)), \; \; m \in \mathbb{R} \end{equation} where

1. $\pi:TQ \to Q$ is the canonical bundle projection, in local coordinates $\pi(q^{i},\dot q^{i})=q^{i}$;
2. $g$ is a Riemannian metric on $Q$, $\frac{m}{2} g(v,v)$ îs the so-called kinetic energy;
3. $V:Q \to \mathbb{R}$ is a smooth function on $Q$, the so-called potential energy.

Note that if we choose $Q=\mathbb{R}^{n}$ with the canonical metric induced by the Euclidean norm, we obtain the well-known formula from physics in coordinates: \begin{equation} L(q,\dot q)=\frac{m}{2} \sum_{i=1}^{n}\limits \dot q_i ^2 - V(q). \end{equation} To be more explicit, we make the specific choice of $n=1$ and moreover:

1. $g$ being the canonical metric induced by the Euclidean norm;
2. $V:Q \to \mathbb{R}, \;\; V(q)=\frac{kq^2}{2}$, where $k$ is a real number.

In this case, we obtain the one-dimensional harmonic oscillator Lagrangian: \begin{equation} L_{1dho}=\frac{m}{2} \dot{q}^2 - \frac{kq^2}{2}. \end{equation} This means concretely that $(\mathbb{R},T \mathbb{R} \cong \mathbb{R}^2,L_{1dho})$ is a Lagrangian system.

We could construct further examples from physics, by choosing a specific function $V$. However, that is not our purpose right now, as we just wanted to elaborate mathematically on the Lagrangian formalism. As a summary, let us conclude that the input data of the Lagrangian formalism consists of a smooth manifold $Q$ and a smooth function $L:TQ \to \mathbb{R}$, called the Lagrangian.

Let's turn now to the Hamiltonian formalism.

A Hamiltonian system is a tuple $(X,\omega,H)$, where:

1. $X$ is a smooth $2n$-dimensional manifold;
2. $\omega: T_a X \times T_a X \to \mathbb{R}$ is a non-degenerate $2$-form for all $a \in X$ and $d \omega=0$, i.e. $\omega$ is closed- that is, it's a symplectic form;
3. $H:X \to \mathbb{R}$ is a smooth function, called the Hamiltonian.

Note that the symplectic form gives rise to dynamics, as it is related to the Poisson bracket via the formula \begin{equation} \{f,g\}=\omega(X_{f},X_{g}), \; \; \text{where} \; \; X_{f},X_{g} \; \; \text{ are the Hamiltonian vector fields associated to} \;\; f,g. \end{equation}

Note, once again, that the input data of Hamiltonian mechanics is a symplectic manifold $(X,\omega)$ together with a smooth function, called the Hamiltonian $H$. Now let us recall the Darboux theorem.

Let $(X,\omega)$ be a symplectic manifold. Then every point $a \in X$ has an open neighbourhood $U \subset X$ and a chart \begin{equation} \varphi=(q^1,q^2,\dots,q^n,p_1,p_2,\dots,p_n):U \to \mathbb{R}^{2n}, \; \; \end{equation} such that in these coordinates, we have: \begin{equation} \omega|_{U}=dq^{j} \wedge dp_{j}. \end{equation} These coordinates $(q^1,\dots,q^n,p_1,\dots,p_n)$ are called canonical coordinates.

Note that the existence of canonical coordinates is a consequence of $(X,\omega)$ being a symplectic manifold! Now, we might ask the question: are these canonical coordinates related to the coordinates $(q^{i},\dot{q}^{i})$ of Lagrangian mechanics in any reasonable manner? So far, we did not make any connection, just wrote down the existence of such, which was implied by the symplectic structure!

Before we answer that question, let us give a whole class of examples known from physics, in this abstract language of symplectic geometry. To this end, we choose $X=T^{*}Q$ for some smooth manifold $Q$. We call $T^{*}Q$ the momentum phase space. In this case, the Hamiltonian becomes a function \begin{equation} H: T^{*}Q \to \mathbb{R}, \; \; H(q^{i},p_{i}) \in \mathbb{R}. \end{equation} As we can see, there's a fundamental difference between the Lagrangian and Hamiltonian, even if they are both smooth functions:

1. The Lagrangian is a smooth function on the tangent bundle of the configuration space, i.e. on the velocity phase space;
2. The Hamiltonian is a smooth function on the cotangent bundle of the configuration space (as a special case), i.e. on the momentum phase space.

So before turning to relating the two, let us give the concrete example of harmonic oscillator. In this case:

1. $X=T^{*}\mathbb{R} \cong \mathbb{R}^2$;
2. $H_{1dho}:\mathbb{R}^2 \to \mathbb{R}, \; \; H_{1dho}(q,p)=\frac{p^2}{2m} + \frac{1}{2} k q^2$, where $k,m \in \mathbb{R}$ are real numbers.
3. $\omega_{1dho}=dq \wedge dp$

Explicitly, this means that the tuple $(T^{*}\mathbb{R} \cong \mathbb{R}^2,\omega_{1dho},H_{1dho})$ is a Hamiltonian system.

Now, as we have seen, that, at least in special case, both formalisms start from a smooth manifold $Q$, and are specified by two functions $L$,$H$ on the tangent and cotangent bundle respectively, in order to go from Lagrangian mechanics to Hamiltonian Mechanics, we would need a map \begin{equation} f:TQ \to T^{*}Q, \end{equation} with help of which, given a Lagrangian on $TQ$, we could produce a Hamiltonian on $T^{*}Q$. This is solved by the fiber derivative.

Let $(Q,M,L)$ be a Lagrangian system with Lagrangian $L:TQ \to \mathbb{R}$. The fiber derivative of $L$ is a map \begin{equation} \mathbb{F}L:TQ \to T^{*}Q, \; \; ((\mathbb{F}L)(v))(w):=\left.\frac{d}{dt}\right|_{t=0} L(v+tw). \end{equation} Clearly, $\mathbb{F}L$ is a smooth function, as $L$ is. In case $\mathbb{F}L$ is a diffeomorphism, we say that the Lagrangian $L$ is hyperregular. The function \begin{equation} E:TQ \to \mathbb{R}, \; \; E(v):=\mathbb{F}L(v)-L(v) \end{equation} is called the total energy of the Lagrangian system $(Q,M,L)$.

Now, suppose the Lagrangian $L$ takes the form introduced before, namely: \begin{equation} L(v)=\frac{m}{2} g(v,v) - V(\pi(v)), \; \; m \in \mathbb{R}. \end{equation} We have that $\mathbb{F}L(v)=m g(v,v)$, since:

1. $\mathbb{F}V(\pi(v))$=0, because $V$ is constant on the fibers;
2. $\left.\frac{d}{dt}\right|_{t=0} g(v+tv,v+tv)=\left. \frac{d}{dt} \right|_{t=0} (g(v,v)+g(v,tv)+g(tv, v)+g(tv, tv))=g(v,v)+g(v,v)=2g(v,v).$

Therefore, the total energy reads: \begin{equation} E=mg(v,v)- \frac{m}{2}g(v,v) + V(\pi(v))=\frac{m}{2} g(v,v)+V(\pi(v)), \end{equation} or in special case of $\mathbb{R}$, in physics notation: \begin{equation} E=\frac{mv^2}{2} +V(q), \end{equation} which is nothing else than the intuitive notion of total energy, i.e. the sum of kinetic and potential energy.

Now let us compare this abstract definition of fiber derivative, to something well-known, the Legendre transform. To this end, we consider $Q \subseteq \mathbb{R}^{n}, TQ \cong Q \times \mathbb{R}^{n}$. In this case, the fiber derivative reads: \begin{equation} ((\mathbb{F}L)(x,v))(w):=(x, D_2 L(x,v)(w)), \end{equation} where $D_2$ denotes partial derivative of $L$ with respect to the second coordinate. If we introduce coordinates $q^{i}$ on $Q$ and $(q^{i},\dot{q}^{i})$ on $TQ$ and $(q^{i},p_{i})$ on $T^{*}Q$,, this reduces to: \begin{equation} (\mathbb{F}L)(q^{i},\dot{q}^{i})=\left(q^{i},\frac{ \partial L}{\partial \dot q^{i}} \right), \end{equation} which lets us read off that $p_i=\frac{ \partial L}{\partial \dot q^{i}}$. Moreover, the thus associated $E$ is the Legendre transform of $L$.

Let us illustrate the above abstract definition of the Fiber derivative in an easier example. Consider the Lagrangian system $(\mathbb{R},\mathbb{R}^{2},L_{f})$ of a free particle, where \begin{equation} L_{f}(q,v):=\frac{mv^2}{2}. \end{equation} In this case, the fiber derivative is a map: \begin{equation} \mathbb{F} L_{f}: \mathbb{R}^2 \to \mathbb{R}^2, \;\; (\mathbb{F} L_{f}(q,v))(w)=m \langle v, w \rangle=\langle mv, w \rangle, \end{equation} so we can identify $(\mathbb{F} L_{f})(q,v) \in T^{*}_{q} \mathbb{R}$ with the momentum $p=mv$.

Now, let us go back into the more general picture. Let $(Q,M,L)$ be a Lagrangian system and choose coordinates $(q^{1},\dots,q^{n}$ on $Q$ and $(q^{1},\dots,q^{n},\dot{q}^{1},\dots,\dot{q}^{n})$ on $TQ$. We define a one-form locally, which is induced by L as: \begin{equation} \lambda_{L}:=\frac{\partial L}{\partial \dot q^{I}} dq^{i}. \end{equation} This one-form is called the Liouville form and gives rise to a two-form: \begin{equation} \omega_{L}:=- d \lambda_{L}= - \frac{\partial ^2 L}{\partial q^{j} \partial \dot{q}^{k}} dq^{j} \wedge dq^{k} - \frac{\partial^2 L}{\partial \dot{q}^{j} \dot{q}^{k}} d \dot{q}^{j} \wedge dq^{k}. \end{equation} I claim $\omega_L$ is well-defined on all of $M$ and it is closed, therefore, it is a symplectic form if and only if it is non-degenerate. We now give a proposition, which tells us when this is the case.

Let $(Q,M,L)$ be a Lagrangian system with Liouville one-form $\lambda_{L}$, which gives rise to the $2-$form $\omega_L$. Then, the following statements are equivalent:

1. $\omega_L$ is non-degenerate;
2. $L$ is hyperregular;
3. $\det \left( \frac{\partial^2 L}{\partial \dot{q}^{j} \dot{q}^{k}}\right) \neq 0 $

We now conclude the relation between Lagrangian Mechanics and Hamiltonian Mechanics with the following proposition:

Let $(Q,M,L)$ be a Lagrangian system with a Lagrangian $L$ that is hyper regular. Then the tuple $\left(T^{*}Q,\omega:=(\mathbb{F}L)^{*} \omega_{L},H:=E \circ (\mathbb{F}L)^{-1} \right)$ is a Hamiltonian system.

In particular, this means that hyperregular Lagrangians $L$ on $TQ$ induce Hamiltonian systems on $T^{*}Q$. Conversely, one can show that hyperregular Hamiltonians on $T^{*}Q$ come from Lagrangians on $TQ$, which lets us conclude:

Not every Hamiltonian system has a Lagrangian formulation. A Hamiltonian system where the momentum phase space is a cotangent bundle admits a Lagrangian formulation precisely when its Hamiltonian is hyperregular. Moreover, there are Hamiltonian systems, which are not cotangent bundles.

References: Martin Schottenloher, Geometric Quantization notes

K.H. Neeb, Physics and geometry notes

Ratiu Marsden, Introduction to Mechanics and Symmetry.

EDIT: A physical remark/add-on, related to your statement that in quantum mechanics one usually starts with a Hamiltonian. This follows from the fact that physicists usually describe QFTs via Path Integrals: the Lagrangian version of the path integral exists only for a specific class of Hamiltonians - see the derivation in Weinberg for this.

EDIT: I will add a concrete physical example, where the condition is not met. Consider the Lagrangian of a relativistic free particle \begin{equation} L_{inv}=-mc \sqrt{ \frac{dx^{\alpha}}{d \tau} \frac{dx_{\alpha}}{d \tau}}=-mc \sqrt{\dot{x} ^2}, \; \; \text{where} \; \; \dot x^{\alpha}:=\frac{dx^{\alpha}}{d \tau}. \end{equation} For the existence of the square root, we must have $\dot{x}^2>0$, i.e. $\dot x$ must be timeline. The Euler Lagrange equations read: \begin{equation} \frac{\partial L_{inv}}{\partial x^{\alpha}} - \frac{d}{d \tau} \frac{\partial L_{inv}}{\partial \dot{x}^{\alpha}}=0. \end{equation} Concretely, computing it, we obtain: \begin{equation} \frac{d}{d \tau} \frac{ mc \dot{x}_{\alpha}}{\sqrt {\dot{x}^2}}=0. \end{equation} The momentum canonically conjugate to $x^{\alpha}$ reads \begin{equation} p_{\alpha}=\frac{\partial L_{inv}}{\partial \dot{x}^{\alpha}}=-mc \frac{\dot{x}_{\alpha}}{\sqrt{\dot{x}^2}}.\end{equation} It satisfies the constraint \begin{equation} p^2-mc^2=0. \end{equation} Note that the dynamics is encoded in this constraint! To see this, we could naively try to construct the Hamiltonian according to the Legendre transformation. We will see this won't work, but let's give it a try. Using the Legendre transform: \begin{equation} H=\dot{x}^{\alpha} p_{\alpha}-L_{inv}=mc \left(-\frac{\dot{x}^2}{\sqrt{\dot{x}^2}}+ \sqrt{\dot{x}^2} \right)=0, \end{equation} which means that the Hamiltonian is the constant zero function, if obtained by the Legendre transformation. The reason for this is precisely the regularity condition of $L_{inv}$. We claim that: \begin{equation} \det \left(\frac{\partial L_{inv}}{\partial \dot{x}^{\beta} \partial \dot{x}^{\alpha}}\right) \neq 0 \end{equation} is not fulfilled. One can explicitly check, using the form of $L_{inv}$ that this is not fulfilled, but I leave the details to the reader. This can be found in Florian Sheck's book, chapter 4.11..

The upshot of this is the following: Passing from Lagrangian to Hamiltonian mechanics is always possible, if the Lagrangian is hyperregular. In case it is not, while passing from one to the other, we lose some data. We must keep track of this data in terms of constraints, we must do constrain analysis, as was pointed out in the comment of a different post by @AcuriousMind.

For a more detailed treatment of constraints for the Lagrangian I provided, please consult this post.

Answer 2

You get $p$ exactly as automatically as $\dot{q}$: Starting with the configuration space of your system, i.e. the space of generalized coordinates $q$, the phase space is the cotangent bundle, and the "correct" coordinates on that phase space are the natural coordinates $(q^i, p_i)$ where given a choice of generalized coordinates $q^i$ we expand a covector $p$ at $q$ as $p = p_i\mathrm{d}q^i$ to get the $p_i$ coordinates.

Answer 3

1. Traditionally the Hamiltonian formulation and the canonical coordinates $(q^1,\ldots, q^n,p_1,\ldots, p_n)$ are derived from the Lagrangian formulation via a Legendre transformation. (The opposite is also possible, cf. e.g. this Phys.SE post.)

2. However, OP seems to instead ask for an Hamiltonian approach that does not rely on a Lagrangian formulation. If this is a correct interpretation of OP's question, here is one: If we are given a global function $H\in C^{\infty}(M)$ (which we will call the Hamiltonian by definition) on a $2n$-dimensional symplectic manifold $(M,\omega)$, then we have a Hamiltonian formulation with a time-evolution equation $$\frac{df}{dt}~=~\{f,H\} +\frac{\partial f}{\partial t} .$$ Canonical coordinates $(q^1,\ldots, q^n,p_1,\ldots, p_n)$ are guaranteed to exist locally by Darboux' theorem. They will however not be unique.

Answer 4

In the interest of describing Hamiltonian mechanics as an independent formulation, I will make no reference whatsoever to a Lagrangian.

The fundamental inputs to a Hamiltonian problem are a symplectic manifold $X$ with symplectic form $\omega$, and a smooth function $H$ called the Hamiltonian which generates the dynamics. Given any observable $F$ (which is taken to be a smooth function $F:X\times \mathbb R\rightarrow \mathbb R$), time evolution is given by $$\frac{d}{dt} F = \{F,H\} + \frac{\partial F}{\partial t}$$ where the Poisson bracket $\{\cdot,\cdot\}$ is defined from the symplectic form as $$\{A,B\}:= \omega^\sharp(\mathrm dF, \mathrm dH)$$ and $\omega^\sharp$ is the "inverse" tensor of $\omega$, whose components are $(\omega^\sharp)^{ij} = (\omega_{ij})^{-1}$.

---

The question remains how one might construct $X$ and $\omega$. If we supply a configuration space $Q$ to model the instantaneous configurations of our system, then there is a canonical way to do this - namely, we let $X=T^*Q$. A coordinate chart $(q^1,\ldots, q^n)$ on $Q$ induces a natural coordinate chart $(q^1,\ldots, q^n,p_1,\ldots, p_n)$ on $T^*Q$, and in these coordinates the canonical symplectic form is given by $\omega = \sum_{i=1}^n \mathrm dp_i \wedge \mathrm dq^i$. This induces the Poisson bracket $$\{A,B\} = \sum_{i=1}^n \frac{\partial A}{\partial q^i} \frac{\partial B}{\partial p_i} - \frac{\partial B}{\partial q^i} \frac{\partial A}{\partial p_i}$$ All that remains is to choose a Hamiltonian function to generate the dynamics, which we may do freely.

On the other hand, on some occasions this is not suitable. In particular, it's clear that such a construction could never produce a compact phase space, which may be of interest when studying classical analogs of quantum mechanical systems with finite-dimensional Hilbert spaces (such as spin systems). Therefore, we can also supply a symplectic manifold by hand, without reference to a configuration space or cotangent bundle.

---

For example, we may let $X= \mathrm S^2\subset \mathbb R^3$ be the 2-sphere of radius $j$ and induce a symplectic form on $X$ as follows. Let $\mathrm{vol}_p = (\mathrm dx \wedge \mathrm dy \wedge \mathrm dz)_p$ be the volume form on $\mathbb R^3$ in Cartesian coordinates. The unit normal vector to $\mathrm S^2$ is given by $n = (x \partial x + y \partial y + z \partial z)/j$. The 2-form

$$\omega= \iota_n \mathrm {vol} = (x \mathrm dy\wedge \mathrm dz + y \mathrm dz \wedge \mathrm dx + z \mathrm dx \wedge \mathrm dy)/j$$

constitutes a symplectic form on $\mathrm S^2$, which we can express in arbitrary coordinates. If we choose e.g. spherical coordinates, we find that

$$\omega = j\sin(\theta) \mathrm d\theta \wedge \mathrm d\phi$$ These are not canonical coordinates because $\omega$ does not take the form $\mathrm dp \wedge \mathrm dq$. That's okay. We can find canonical coordinates (as guaranteed by the Darboux theorem), which in this case is given by defining $z = j\cos(\theta)$ whereby we find that $$\omega = \mathrm d\phi \wedge \mathrm dz$$ These are just cylindrical coordinates. Writing down a Hamiltonian function then prescribes the dynamics of the system. Imagining a magnetic moment in an external field, we might let $H = -\omega_0 z$ in which case we have that $$\frac{d}{dt} z = \{z,H\} = 0 $$ $$\frac{d}{dt} \phi = \{\phi,H\} = \omega_0 $$ the solutions of which describe a classical analog of a quantum spinor rotating about the $z$-axis with frequency $\omega_0$.

Incidentally, the geometrical quantization procedure dictates that we take our phase space $X$ as the base space for a Hermitian line bundle whose connection $\mathcal A$ has curvature $\mathrm dA = i \omega$. However, the quantization of the Chern number (see e.g. here) implies that $\oint_X\omega = 4\pi j \in 2\pi \mathbb Z$ - implying that $j\in \mathbb Z/2 = \{0, 1/2, 1, \ldots\}$.

Answer 5

I have myself also looked high and low for the solution to this problem. Other than guesswork, almost everybody works by starting from the Lagrangian, and using the Legendre transform to get the momenta. Just like with the Laplace transform, we build up a library of known pairings and move on from there.

There are a few rare cases whereby people made systems that are obviously without a Lagrangian yet has a Hamiltonian. Those seem to all come from guesswork. It is not clear if there is a systematic way to get from laying down a coördinate system, to getting the momentum expressions that can satisfy the symplectic conditions.

Answer 6

So you can kind of use the electromagnetic case as an impossibility result, if you like.

In terms of just a charged particle with charge $q$, position $\mathbf r$, dealing with an external electromagnetic field $(\varphi,\mathbf A)$, the total energy would be written without reference to the proper generalized momentum as simply $$ H(\mathbf r, \dot{\mathbf r}) = \frac12 m~ \dot{\mathbf r}^2 + q~\varphi(\mathbf r). $$ I am calling this $H$ because it is the “Hamiltonian” you will get from doing this the proper Lagrangian way and then substituting $\mathbf p$ as a function of $\dot{\mathbf r}$ but it is not a function of phase space—see comments below.

Notice that the magnetic field $\mathbf A$ makes absolutely no appearance in this energy expression, roughly physically because magnetic fields do not do work and the “Hamiltonian” reflects the total energy. And this expression also is extremely easy to interpret as such, it's literally kinetic energy plus potential energy, what recipe could be simpler? So if you didn't know the right thing to do, the right Lagrangian, you would write this down and it would be “correct” but it wouldn't be obvious how to add the dynamics due to the magnetic field.

So that's the impossibility result: there are several different valid derivations from this expression to the actual equations of motion, each corresponding to a different magnetic field and how that impacts the actual motion of the particle. There is simply no amount of staring at $\mathbf r$ and $\dot{\mathbf r}$ directly that will tell you what $\mathbf A$ is.

This is actually a fundamental theoretical problem as well, if you think of $(q,p)$ as phase space, $H$ is going to be a scalar field over phase space and the Hamiltonian dynamics will confine us to a hypersurface of this phase space... But if you can choose an arbitrary $p(q, \dot q)$ over this hypersurface, like, you can't choose the surface but you have a ton of possibilities for what the dynamics might look like inside that surface, because the motion with respect to q depends on the current definition of p but you have a lot of freedom there.

So what other answers are hinting at, is that you might need to watch the actual dynamics at play as well as knowing the form of the total energy. The simplest part of working out the dynamics is the first Hamilton’s equation, $$ \dot x_i = {\partial H\over\partial p_i} =m\dot x_i{\partial \dot x_i\over\partial p_i}, $$ because we know the independence of the potential $\phi$ and the $p_i$ that we are using for this derivative, at least in this electromagnetic case, and also because we have $\dot x_i$ on both sides here, so clearly we just want to find a constant $\partial \dot x/\partial p = m^{-1}$, and thus turn this complicated expression into an identity. And so in the usual case if we avoid coupling the different velocities with each other, we’ll be trying the ansatz, $$ p_i \overset?= m\dot x_i + g_i(x_1\dots x_n),$$ having the usual momentum term plus a term that varies in space.

This then couples everything together on the momentum side, $$\begin{align} {\mathrm dp_i\over \mathrm dt}&= -{\partial H\over\partial x_i}\\ &=-\sum_j m\dot x_j{\partial \dot x_j\over\partial x_i} - \nabla_i\varphi, \end{align} $$these derivatives being taken at constant $x_j, p_j$. Look at this! Complexities on both sides of the equation, because the left hand side needs a total derivative, which in our ansatz is, $$ {\mathrm dp_i\over \mathrm dt} = m \ddot x_i + \sum_j (\nabla_j g_i)~\dot x_j $$ while the right hand side has this $\partial \dot x_j/\partial x_i$ term which becomes $-m^{-1} \nabla_i g_j,$ the exact reverse combination. So you have actually in this ansatz, $$ m\ddot x_i = -\nabla\phi +\sum_j \dot x_j(\nabla_i g_j - \nabla_j g_i).$$ This very pretty anti-symmetry in this term is probably due to the pretty antisymmetric aspects of the underlying “symplectic form” mentioned in Qmechanic’s answer, but my degree was an applied physics so that's beyond my training.

Note that if we have multiple particles the sum in principle ranges over all of them, this particle’s $\ddot x_i$ might depend on some other particle’s $\dot x_j$. If we try to remove this, we have particles $\mathbf r_i$ and we refine the ansatz to $\mathbf p_i = m \dot{\mathbf r}_i + \mathbf g_i(\mathbf r_i)$ so that most of these $\nabla_i g_j$ terms disappear.

If we then assume that all of the $\mathbf g_i$ are basically the same, $\mathbf g_i(\mathbf r) = q_i~\mathbf A(\mathbf r),$ we finally get an external-magnetic field type of expression.

So those are the sorts of complexities that you can expect if you have $H(x,\dot x)$ with no $L$ and start trying to figure out a suitable $p$ for the dynamics, this Hamiltonian does not quite give you enough information and you need to supplement with other stuff which ends up (through complicated derivations!) to have this very pleasing anti-symmetry in the actual equations of motion, and in that sense you will usually get “the obvious solution (the one when $g=0$) plus some magnetic-ish terms.”