Why gauge theories have such a success?

Question

[This question was inspired by a identical question asked on a other forum]

Note that we may morally include general relativity in the gauge theories.

We may have several (some are deliberately contradictory) remarks :

1) Gauge invariance is only a mathematical redundancy, which is unphysical. In fact, we are not using the correct mathematical tools, and/or we have to give up the standard view of space-time (and action).

2) We can't avoid the gauge formalism, because it corresponds to something fundamental, it is not only more pratical for calculus.

3) For massless particles, it is interesting to consider the helicity formalism (the masslessness derives automatically from the formalism).

Answer 1

I think your remark 1 is right on, and so I disagree with remark 2.

I do not believe that you should consider the gauge formalism as fundamental or even unavoidable. Gauge degrees of freedom arise when you add in fake degrees of freedom in order to make the formulation of the theory simpler or more manifestly symmetric. For instance, we want the quantum theory to be Lorentz invariant and unitary. The representation theory of the Poincare group here tells you that a massless helicity-1 particle has 2 on-shell degrees of freedom. But we also want the theory to be local, and so we use a Lagrangian (off-shell) description to ensure this.

In order to write down a manifestly Lorentz invariant local lagrangian, we must embed the two degrees of freedom in a larger field and use a gauge-invariant lagrangian. This is a choice. We do this because we want to make the theory look explicitly local and Lorentz invariant at every step along the calculation. The gauge redundancy arises from the conflict between unitarity (2 d.o.f) and locality/Lorentz invariance (have to use the gauge field). If we did not want a Lagrangian description (say we had a way of calculating using only on-shell data such as momentum and helicity) or we don’t care if the intermediate steps look Lorentz invariant (i.e. just fix the gauge and calculate) then we could avoid the gauge redundancy.

In fact, as far as I am aware, the people working on N=4 SYM amplitudes are able to do this. They achieve great simplicity in the answers because they work with only on-shell data, not worrying about locality and thus avoiding the gauge redundancy. In my opinion, the easiest way to see such a simplification is to calculate some gluon scattering amplitude using the Feynman rules derived from a Lagrangian (even for small numbers of gluons, the number of diagrams becomes unmanageable because so many are related to each other via gauge redundancy) and then redo the calculation using the spinor-helicity formalism, which allows you to do the calculation on a page and doesn’t make use of any gauge fields, etc.

So in this sense, gauge theories are successful because the forces in nature are mediated by massless helicity-1 (or 2 for gravity) particles, and physicists like/need lagrangian descriptions because they want to make certain properties like locality and Lorentz invariance manifest. Gauge invariance isn’t a fundamental part of nature, it is a relic of the particle content of the universe and the particular way we choose to describe the physics.

Answer 2

As an outsider to particle and BSM physics, I feel a little weird suggesting this to you, Trimok (with what I have gathered about your background and ability from your answers and questions), but here is how I like to see it.

For many years I felt like tearing my hair out whenever I saw a Lagrangian whilst trying to get some layperson's insight into what other physicists were doing, particularly with BSM physics. Not that I had any problem with the mathematical correctness of what was being presented - it's physical grounding just seemed thoroughly mysterious. As Roger Penrose says somewhere in "The Road to Reality" Lagrangian formulations of a theory are "a dime a dozen" (not exactly the words he used: you can dream up a Lagrangian account of any physical theory. How on Earth does one dream up a Lagrangian? It is generally hard if not impossible to look at the terms in a Lagrangian and say "that one means such and such" as you can with many (not all, mind you) physical theories. Penrose made the cryptic comment that the standard model would look thoroughly "contrived" if it weren't for its experimental grounding. I read "contrived" as meaning "not at all physically obvious" but also "if it weren't for ..." implied that the experimental results said this was just how it had to be. I wondered what kind of experimental results would motivate something so abstract as some of the Lagrangians I came across in a way as powerfully as Penrose implied.

Then the following suddenly dawned on me (I think this is what Frederic Brünner's answer is also getting at):

Lagrangian dynamics + Noether's Theorem = A tool for experimentalists to encode their observations into a candidate theory for the theorists to work from

Noether's theorem is of course about Lagrangians, their continuous symmetries and corresponding conserved quantities, exactly one for each continuous symmetry, whose conservation can be described by a corresponding continuity equation. So, if we experimentally find that there are some measured, real-valued quantities which are conserved throughout experiments, let's say "twanglehood", "bloobelship" and "thwarginess", and then a possible theory is one derived from a Lagrangian which is explicitly constructed with one continuous symmetry for each of these. Moreover we might be lucky, as in Frederic Brünner's example to also have three observed continuous symmetries as well. This is now a really strong experimental motivation: we must now write down Lagrangian with the three observed symmetries and try to fit each continuity equation implied by Noether's theorem to "twanglehood", "bloobelship" and "thwarginess", in keeping with whatever else we can experimentally learn about these three.

Once I understood this, then the other mystery melted away. Why do we want physical theories with gauge symmetry - i.e. redundancy in them? Surely physics aims to make things as simple as it can, particularly if the gauge symmetry is not an experimentally initially obvious symmetry of the system? Of course, in the Lagrangian formulation symmetry is needed to beget conservation, so we take on "redundancy" - gauge symmetry - to express that conservation mathematically in a gauge theory.

Since I am not a routine user of these ideas, there is bound to be more to a full answer to your question than my meagre knowledge can put forward, but the above ideas have to be at least a partial answer.

Footnote: I deliberately used the word "continuous" rather than differentiable symmetry: you don't need to assume the latter. A "continuous" symmetry implies a Lie group of symmetries, and the Montgomery, Gleason and Zippin solution to Hilbert's fifth problem shows that $C^0$ assumptions in Lie theory imply an analytic, i.e. $C^\omega$ manifold. I had to get that one in somehow, as a Lie theory enthusiast.

Answer 3

The point of view that gauge symmetries may simply be quotiented out is too simplistic. It is of course true that in scattering theory where one only globally considers something going into a black box and emerging at the end, only the gauge equivalence classes of in- and out-states matter. But what happens inside that black box is local field theory, and there is no way to preserve locality while locally quotienting out gauge transformations. This follows from a simple argument, for exposition see these notes.

In partciular, as discussed there, if one insists that gauge transformations witness just a redundancy also locally, then all instanton sectors disappear, hence then the proper QCD vacuum disappears.

There are more drastic examples. For instance the Wess-Zumino-Witten model arises at the boundary of Chern-Simons theory such that its field configurations (maps to a Lie group $G$) are identified with the boundary gauge transformations of the $G$-Chern-Simons gauge theory. This is a drastic example of gauge transformations not being a redundancy.

In general, it serves to not be too naive and apply a minimum of mathematical sophistication when considering gauge symmetry in fundamental physics. For further reading, see also the exposition Examples of prequantum field theories I: Gauge fields.

Answer 4

One reason gauge theories are successful might be that gauge transformations are not some exotic mathematical curiosity which affects some side branch of a physical concept, but stands at the core of the theory itself: it affects the dynamics of the theory directly by determining the shape of the Lagrangian, from which one can derive many quantities relevant for comparison with experiment (equations of motion, scattering amplitudes). Phenomenology and fundamental principles which are assumed affect the theory in a dramatic way. The principle of gauge symmetries merges flawlessly both with the old and established concepts of classical mechanics and the relatively new and modern theory of quantum mechanics.

Take for example QCD - the assumption and observation that nature does not distinguish between states of different colour decomposition directly leads to the introduction of an $SU(3)$ colour symmetry with all its beautiful and puzzling consequences.