36

Is there any link between the law of maximum entropy and the principle of least action. Is it possible to derive one from the other ?

Qmechanic
  • 201,751
Anarchasis
  • 1,343

4 Answers4

17

Yes, there is a link, both are examples of extremum principles. And yes it is possible to derive the principle of least action from the law of maximum entropy. The derivation is lengthy and I will only sketch the main steps needed:

  1. We start from the law of maximum entropy $dS/dt \geq 0$. As we know this law is only valid for isolated systems [i]. For dissipative systems $dS/dt > 0$, the evolution is irreversible and cannot be described by an action principle. We must consider non-dissipative systems, for which $dS/dt = 0$. This is correct because the action principles are rigorously restricted [ii] to Nondissipative Systems.

  2. From the phase space structure we can show that the phase space state $\rho$ satisfies the equation $d\rho/dt = \partial\rho/\partial t - \mathcal{L}\rho$, where $\mathcal{L}$ is the Liouvillian.

  3. From the constancy of entropy (1), we can derive the Liouville theorem $d\rho/dt=0$, using the Gibbs relation $S=S(\rho)$. This implies that the general equation of motion (2) reduces to the Liouville equation $\partial\rho/\partial t = \mathcal{L}\rho$. Effectively, this equation is not dissipative and conserves entropy.

  4. For a mechanical system in a pure state, the phase space state is given by the well-known product of Dirac deltas; substituting this $\rho_\mathrm{pure}$ on the Liouville equation, the equation reduces to the Hamilton equations of motion: $dq/dt = \partial H / \partial p$ and $dp/dt = -\partial H / \partial q$.

  5. Using the Hamilton Jacobi method, the Hamilton equations of motion can be written again as a single equation: the Hamilton Jacobi equation $H + \partial A / \partial t = 0$, where $A$ is the action [iii].

  6. It can be shown that the Hamilton Jacobi equation "is an equivalent expression of an integral minimization problem such as Hamilton's principle", and Hamilton's principle is just the Hamiltonian version of the principle of least action. In other words, solving the Hamilton Jacobi equation one obtains the action $A$ and this automatically satisfies the principle of least action $\delta A=0$.

  7. Other versions of the principle of least action can be obtained from here. For instance, the Lagrangian version of the principle can be obtained using a Legendre transformation for deriving the Lagrangian $L = pv - H$. In this case, the action is given by $A=\int L dt$.

[i] For non-isolated system entropy can increase, decrease or remain constant.

[ii] As reported in the Scholarpedia link there are some few special dissipative systems which can be described by an action principle. Those are open systems for which the production of entropy is compensated by an external flow of entropy to give a zero total variation $dS=d_iS+d_eS=0$. Moreover, the action principle only describes the average behaviour of these systems, but not the thermal fluctuations.

[iii] References on mechanics usually denote the action by $S$, but here it would be confused with entropy.

juanrga
  • 5,645
  • 2
    I don't know how you derive Liouville's Theorem from the constancy of the entropy. Further, you have already limited the discussion to an isolated system and the entropy is more general than that. – Paul J. Gans Dec 26 '12 at 19:51
  • dS=0 is only a special (non-equilibriym) case of isolated system. – unsym Dec 26 '12 at 20:09
  • 1
    @juanrga: Both your accounts are unregistered. You may need to add a login (click "my logins" on your profile) to make it a registered account. Unregistered accounts expire with the browser session. If you want your accounts to be merged, add "merge with " on both profile pages – Manishearth Dec 26 '12 at 20:36
  • 2
    @PaulJ.Gans $dS/dt = -k\int dp dq (\ln \rho + 1) d\rho/dt$. Yes I limited the discussion to isolated systems, because the OP asked for derivation from "the law of maximum entropy" – juanrga Dec 27 '12 at 19:09
  • @Manishearth I could not merge, but moderators help me to recover my previous account. Thank you! – juanrga Dec 27 '12 at 19:12
  • @juanrga: Ah, I see :) It seems like you added a login, finally. I wasn't willing to merge initially because there was insufficient evidence that your two accounts were the same. – Manishearth Dec 27 '12 at 19:16
  • 1
    Technically everything seems to be correct and indeed answers half of the OP's question. – Yrogirg Dec 30 '12 at 17:52
  • 1
    Concerning the second point --- how to you derive the form of $\mathcal L$? Shouldn't you here effectively postulate Hamiltonian mechanics? – Yrogirg Dec 31 '12 at 03:51
  • 3
    @Yrogirg Good point! From the analysis of the total differential $d\rho$ in phase space one finds that the $\mathcal L \rho$ is a flow term in phase space. This flow involves a function $H(p,q;t)$ that resembles the Hamiltonian of Hamiltonian mechanics $H(p(t),q(t))$, but is not exactly it. The correct term is Liouvillian mechanics. Hamiltonian mechanics is obtained as a special case when $H(p(t),q(t)) = \langle H(p,q;t) \rangle$. – juanrga Dec 31 '12 at 12:12
  • 1
    This answer is misleading. Least action is incompatible with entropy increase. All this answer shows is that if entropy does not increase, the incompatibility disappears. – Johannes Jan 03 '13 at 15:27
  • 2
    @Johannes No. The OP asked about "the law of maximum entropy" and my answer starts from this law: $dS/dt \geq 0$. Precisely the entropy is a maximum when the equal sign applies. – juanrga Jan 04 '13 at 15:20
  • 1
    Your answer starts with the statement "We start from the law of maximum entropy dS/dt≥0." That is different from what you now state. In any case, the only relevant physical principle is the 2nd law of thermodynamics. – Johannes Jan 04 '13 at 16:22
  • 1
    @Johannes How "We start from the law of maximum entropy $dS/dt\geq0$." is different from "The OP asked about 'the law of maximum entropy' and my answer starts from this law: $dS/dt\geq0$." 8-0 – juanrga Jan 05 '13 at 17:41
  • 1
    @juanrga could you give some written reference about the whole derivation? Thanks – J L Feb 04 '13 at 10:36
  • @Nivalth I don't know. – juanrga Feb 14 '13 at 19:34
  • +1, very nice, one note, for a dissipative system one can still use maximum entropy principle (and thus derive a stationary action principle) by enlarging the scope or space . For example considering an ensemble or similar, and passing to density description of the system – Nikos M. Jun 17 '14 at 17:13
  • some related links, here and Jaynes's maxent and action, also check the generalised formulations of entropy (2dd law) for both equilibrium and non-quilibrium thermodynamics – Nikos M. Jun 17 '14 at 17:20
  • 1
    @juanrga, I do not think you have answered the question how you derive the Liouville equation from $d/dt\int -\rho\log \rho ,dqdp = 0$. I do not think that is possible. It is the other way around, from the Liouville equation we can derive $d/dt\int -\rho\log \rho ,dqdp = 0$. – Ján Lalinský Jun 30 '15 at 20:59
  • Interesting recent paper on this subject: https://arxiv.org/abs/1509.08943 – Rococo May 13 '16 at 03:28
  • @juanrga How do we conclude $\frac{\mathrm{d}\rho}{\mathrm{d}t}=0$ from $\frac{\mathrm{d}S}{\mathrm{d}t}=−k\int,\mathrm{d}p,\mathrm{d}q,(ln\rho+1) \frac{\mathrm{d}\rho}{\mathrm{d}t}$? I can conceive of the integral being zero, without the integrand necessarily being identically zero. Am I missing something? – Sanha Cheong Jul 04 '17 at 15:26
  • @SanhaCheong. S above is the local phase-space entropy. Integrating on momenta variables gives the local space entropy used in TIP. Further integrating on position variables gives the entropy of classical thermodynamics. Your phase space integral is what I would denote by . – juanrga Jul 05 '17 at 11:15
  • @juanrga but to PaulJ.Gans's question regarding deriving Liouville's Theorem from constancy of $S$, you wrote that equation. Am I misunderstanding something? – Sanha Cheong Jul 06 '17 at 07:56
  • @SanhaCheong. I see. I cannot edit that now. Change it to $dS/dt = -k (\ln \rho + 1) d\rho/dt$. – juanrga Jul 06 '17 at 08:56
  • @juanrga I am dubious about 2. Isn't the phase space structure already assuming the underlying existence of the action? – Juan I. Perotti Jan 15 '22 at 06:25
  • 1
    @JuanI.Perotti the phase space structure is also valid for dissipative systems whose equation of motion cannot be obtained from any action. – juanrga Jan 19 '22 at 20:12
  • @juanrga could you, please, provide a reference? – Juan I. Perotti Jan 21 '22 at 01:37
  • 1
    @JuanI.Perotti I do not have the reference at hand, but I recall reading in classic Prigogine's monograph (thermodynamics of irreversible processes) something about how he and his group found that the equations for dissipative structures could not be found by extremum techniques. Maybe my memory fails, I will try to get the reference to check it. – juanrga Jan 24 '22 at 13:41
9

"Entropy" and "action" are two entirely different concepts. The first relates to a coarse-grained statistical description of a physical system at macroscopic scales, the latter to the underlying deterministic microscopic dynamics exhibited by the system.

Also note that: 1) the second law of thermodynamics tells us entropy doesn't decrease, it need not increase and certainly can attain non-maximal values, and 2) action is stationary and not necessarily minimal or maximal. Hence, when considered as fundamental physics laws, both 'maximal entropy' and the 'principle of least action' are misnomers.

Zooming in to the core of your question: 'entropy increase' and 'action stationarity' are unrelated, and even incompatible. One certainly can not be derived from the other. This is for the simple reason that 'action stationarity' describes a reversible physics, while 'entropy increase' presents us with an irreversible picture of evolution of physical systems. The difference, again, is in microscopic versus macroscopic.

As an analogy, think about two statements one can make about the physics of pool billiard. The first being that the balls collide according to Newtons laws which can be expressed by stating that the detailed balls trajectories render a quantity called 'action' stationary. The second being the coarse-grained statistical statement that as long as balls aren't pocketed yet, the mean distance between the balls doesn't decrease. Both statements are unrelated and apply to a description of pool billiard at different levels.

Johannes
  • 19,015
  • 2
    There are many issues in this answer. For instance, it confounds the general concept of "entropy" with the approx. macroscopic entropy used in classical thermodynamics. The entropies used in nanothermodynamics, thermodynamics of small systems and in quantum thermodynamics are not defined "at macroscopic scales"... – juanrga Dec 30 '12 at 14:15
  • 2
    Not sure what exact problems you have with my answer. Reading it again, I have used the oportunity to expand it significantly. Let me know in case you believe anything in my answer is incorrect or didactically sub-optimal. – Johannes Dec 30 '12 at 16:51
  • 1
    As said "there are many" I gave one example in my previous post, which you have ignored. Your edit adds more misunderstandings and false statements. E.g., your recent "the second law of thermodynamics tells us entropy doesn't decrease" is a typical misunderstanding of the second law repeated by anti-evolucionists for instance... – juanrga Dec 31 '12 at 11:59
2

As Joannes says, the two principles belong to two different theories:

  • the least action principle is a principle about the (conservative) laws of motion and a proposition about the paths actually followed by the degrees of freedom of mechanical bodies

  • the maximum entropy principle refers either to thermodynamics to figure out in which direction will a transformation occur or more generally to Bayesian inference theory and is rarely concerned with dynamics

They are thus quite different beasts conceptually and, although there may be some overlaps between the two in some instances, I would not recommend, as a matter of principle, to think that these two things are closely related as it is not the case.

As for "deriving" one from the other, at the very least we should agree first about what are the axioms we start from.

For instance, some versions of Crook's fluctuation theorem enable one to get something very close to a maximum entropy principle (on average) "from" simply hamiltonian mechanics (which I would put on the same footing as the least action principle for now) but it is clear that the theorem relies a lot on probability theory (and possibly on Bayesian inference) which, in my view at least, is outside the scope of the least action principle alone.

I think that's an important shift to notice even when invoking Liouville's theorem which is a theorem that makes propositions about probability densities and not about trajectories.

gatsu
  • 7,202
2

This old russian paper seemed to have discussed the problem without going much into non-equilibrium statistical mechanics. I haven't read the entire thing. Soviet Physics Journal May 1991, Volume 34, Issue 5, pp 426-431. Moslov is the author. Check out their other papers too.

MarcelineH
  • 520
  • 3
  • 14