How would someone discover the Einstein-Hilbert Action?

Question

Usually in textbooks or on online resources, when you are learning General Relativity, propose the following: $$ \mathcal S[g_{\mu\nu}] =\frac{1}{2 \kappa}\int_{\mathcal M} {\mathrm d^4 x \; R\sqrt{-g}} $$ This is the Einstein-Hilbert Action. All textbooks, in which appears, propose it as something trivial, but how Hilbert arrived to this conclusion? The best I could do, while searching for information, was to find how to vary the action and find Einstein-Field Equations in the vacuum from it. But my question is: what procedure did Hilbert follow to find this formula, how would you derive it (and know this will give EFE in the vacuum)?

Answer 1

I don't know the history of how the Einstein-Hilbert action was discovered originally, but from a modern point of view it can be justified in several ways.

First, if you want to represent gravity as geometry of spacetime, and you don't want a fixed "background" geometry, then you need the action for pure gravity to be a scalar built out of the metric and derivatives. By locality, we also want the action to be an integral of a local function. The logic of effective field theory (see also this review) tells us that at low energies, the terms in the action with the smallest numbers of derivatives will be the most important. Then...

• At zeroth order in derivatives, the only local scalar function involving the metric is a constant (the cosmological constant) \begin{equation} S_{CC} = \int {\rm d}^4 x \sqrt{-g} \end{equation}
• At first order in derivatives, there are no candidate scalar functions. With one derivative, you would have an unpaired index $\nabla_\mu$ which has nothing to be contracted with.
• At second order in derivatives, there is one candidate scalar function, the Ricci scalar $R$. This immediately gives us the Einstein-Hilbert action \begin{equation} S_{EH} = \int {\rm d}^4 x \sqrt{-g} R \end{equation}
• At fourth order in derivatives, there are three candidate terms, which could appear, generally with arbitrary coefficients \begin{equation} S_{4} = \int {\rm d}^4 x \sqrt{-g} \left(c_1 R^2 + c_2 R_{\mu\nu} R^{\mu\nu} + c_3 R_{\mu\nu\rho\sigma} R^{\mu\nu\rho\sigma} \right) \end{equation}
• There are more possible combinations at higher orders in derivatives (also including covariant derivatives acting on the Riemann curvature tensor) but we expect these to be relatively unimportant (or irrelevant in the lingo) at low energies.

So the Einstein-Hilbert term plus a cosmological constant is a natural guess for a low energy theory of gravity from modern effective field theory logic.

There are also more sophisticated methods to justify the Einstein-Hilbert action. In particular:

• The Einstein-Hilbert action is the unique low energy theory of an interacting, Lorentz invariant, local, massless spin-2 particle. This is justified in a series of works, including the Feynman lectures on gravitation, and this classic paper by Deser: https://arxiv.org/abs/gr-qc/0411023.

• In order to have second order equations of motion (which is generally required to avoid ghost instabilities), only special combinations of curvature tensors can be used in the Lagrangian, called Lovelock invariants. The Einstein-Hilbert term (and cosmological constant) are the simplest (lowest order) Lovelock invariants. (And in four dimensions, the only non-trivial Lovelock invariants).

  • There is also $f(R)$ gravity, where you have an arbitrary function of the Ricci scalar as your action, but this describes a spin-2 particle plus an additional scalar degree of freedom, so adds an extra physical mode relative to general relativity.

Answer 2

Einstein argued that physical laws should be generally covariant. This can be physically motivated from the argument that the physics of a situation should not be dependent upon how we describe the situation through coordinates. Formalising this leads to the notion of spacetime being described by manifolds and physical laws being covariant wrt to diffeomorphisms of the spacetime manifold. The natural structure on the manifold, after Minkowski's reformulation of SR as a 4d theory of spacetime, is a Lorentzian metric and the natural invariant of this is its curvature.

It's through these and other arguments that Einstein discovered the field equations of GR.

Hilbert took a different route. He searched instead for an action principle. The simplest Lagrangian or action density here is the scalar curvature. When he looked at its Euler-Lagrange eom he discovered Einstein's field equatuons roughly about the same time.

It also turns out that this is the correct action to unify the classical theory of gravity and YM. Roughly speaking, if we look at principal bundle with structure group G over the spacetime manifold amd impose the same action then the variations wrt metric gives Einstein's field equations whilst varying the connection yields the Yang-Mills equations. All this came out of Kaluza, Klien and Nordstrom's work on higher dimensional gravity.

Answer 3

A physicist is liable to build a theory as the simplest option that does what they want. The truth is we can't "deduce" a unique theory; @Andrew's answer gives a great overview of more complex alternatives. Let's see how far this KISS principle takes us.

One of the earliest findings when working with covariant derivatives on a differentiable manifold is they don't commute. Even under the zero-torsion assumption of general relativity - which fits the data (if there is torsion it's very small), simplifies and uniquely determines such derivatives - we get $[\nabla_a,\,\nabla_b]V_c=R_{abcd}V^d$, with $R_{abcd}$ the Riemann tensor. (If you add in torsion, we get another interesting coefficient tensor, but let's keep it simple.)

Since this tensor quantifies how far we depart from partial derivatives' famous commutativity, and it's made of the metric tensor and derivatives up to its second, scalars made from it are natural candidate terms in the scalar Lagrangian density (a little-used term for Lagrangian density divided by $\sqrt{|g|}$). Next we keep it "simple" in the sense of being linear. The tensor turns out to have such antisymmetries we only have one linear option, namely the Ricci scalar $R=R_{abcd}g^{ac}g^{bd}$.