What are quantum fluctuations, really?

Question

So, I have read a lot about quantum fluctuations, and how they are responsible for:

• Lamb shift
• Spontaneous emission of photons from excited atomic states
• Casimir effect
• ...

and the explanation always mentions either or both of these two things:

• $E$ being the energy, $\langle E\rangle = 0$ but $\langle E^2\rangle \neq 0$ because of the $\Delta E \Delta t > \hbar$ uncertainty principle, so actually there is some temporal appearance of energy
• The vacuum is full of pairs of virtual particles and antiparticles annihilating each other, and for a time $\Delta t$ dictated by the above uncertainty principle, they can interat with real particles.

---

Now, I fall in the line of though that:

• $\Delta E \Delta t > \hbar$ is not a real uncertainty principle, as t is not an operator. This expression is derived from Ehrenfest theorem and quantifies the maximum change that the energy eigenvalue can undergo in an interval $\Delta t$.
• Virtual particles do not exist, they are just an artefact of perturbative expansions.

---

Is there a physical, QFT-free, qualitative way to understand quantum fluctuations? And in order to make sense of the very real phenomena associated with them, like the ones I listed at the beginning?

Is there a relation to the ground state energy problem leading to the cosmological constant problem?

Answer 1

You asked for a qualitative picture, so here goes.

Consider a simplified example: the quantum harmonic oscillator.

Its ground state is given by

$$ \Psi(x) = \text{const} \cdot \exp \left( - m \omega_0 x^2 / 2 \hbar \right). $$

Now suppose that we are measuring the position of this oscillator in the ground state. We could get any real value, with probability density $|\Psi|^2$. In reality, because of the exponential decay, most of the values are distributed within the window of width

$$ \Delta x \sim \sqrt{\frac{2 \hbar}{m \omega_0}}, $$

with the mean concentrated at $x = 0$.

Because measuring an individual oscillator is a complicated process which results in it getting entangled with the measurement device, let's simplify the problem – say we have an ensemble of non-interacting oscillators all in ground states, and we measure them all independently. The distribution of values $\{x_i\}$ is expected to mostly lie within the mentioned above window, but the actual values are unknown. We usually say that those are due to quantum fluctuations of the position operator.

The same thing happens with the quantum field, which upon inspection is nothing more than a collection of weakly interacting harmonic oscillators. If we take an ensemble of vacuum quantum field configurations (say, independent experiments at a particle accelerator), and we measure a value of the field at a point, we will see that it is not equal to zero (as it would be in the classical theory), but instead the values are distributed within an error window and are otherwise random. This are quantum fluctuations of the QFT vacuum.

These fluctuations are sometimes attributed to "virtual particles", or "virtual pairs", which are said to be "born from the vacuum". Sometimes it is also said that they can "borrow energy from vacuum for a short period of time". AFAIK these are just analogies, appealing to the consequence of Erenfest's theorem (the so-called time-energy uncertainty relation).

But the fluctuations undisputably have very real, measurable effects. Qualitatively, those effects come from a difference between the physical picture of the same thing painted by classical fields and quantum fields. You can say that quantum fields reproduce classical fields on certain scales (measured in the field value), which are much greater than the size of the error window. But once the precision with which you measure field values becomes comparable to the size of the error window, quantum effects kick in. Those who like painting intuitive pictures in their heads say that this is caused by quantum fluctuations, or virtual particles.

UPDATE

Belief that observed Casimir effect has something to do with vacuum fluctuations of the fundamental QFT is misguided. In fact, in the calculation of the Casimir force we use an effective field theory – free electromagnetism in the 1D box, bounded by the two plates. Then we look at the effective vacuum state of this effective QFT, and we interpret the Casimir force as a consequence of the dependence of its properties on the displacement between the plates, $d$.

From the point of view of the fundamental QFT however (Standard Model, etc.) there is no external conducting plates in the first place. If there were, it would violate Lorentz invariance. Real plates used in real experiments are made of the same matter described by the fundamental QFT, thus the state of interest is extremely complicated. What we observe as Casimir force is really just a complicated interaction of the fundamental QFT, which describes the time evolution of the complicated initial state (which describes the plates + electromagnetic field in between).

It is hopeless to try to calculate this in the fundamental QFT, just like it is hopeless to calculate the properties of the tennis ball by studying directly electromagnetic interactions holding its atoms together. Instead, we turn to the effective description, which captures all the interesting properties of our setup. In this case it is free electromagnetic effective QFT in the 1D box.

So to summarize: we are looking at the vacuum state of the effective QFT and the dependence of its properties on $d$. Alternatively, we are observing an extremely complicated fundamental system in a state which we can't hope to describe.

Answer 2

I completely share your frustration that people often describe very complicated and precise results as coming from "quantum fluctuations" without ever defining what that term actually means. After several years of hearing the term thrown about, I've come to the conclusion that "quantum fluctuations" is simply synonymous with a state being in a superposition of classical states (e.g. position eigenstates for a particle, or product states for a spin system).

We often work in a semiclassical regime where the state of interest is in a superposition that is strongly weighted toward a single classical state (or a narrow range of "similar" classical states). Then we can think of the system as "mostly" being in that dominant classical state, but with "quantum fluctuations" that result in our occasionally measuring something other than that dominant value due to the Born rule. But sometimes (e.g. in strongly coupled quantum systems) the state of interest is a fairly uniform superposition over a very wide range of different classical states, so the "single classical state plus small quantum fluctuations" picture is no longer useful.

Another thing that might be helpful to note is that people often think of quantum superpositions by analogy with thermal mixtures of different states in statistical mechanics. So when they talk about "quantum fluctuations" they are making an analogy with thermal fluctuations, where there is always a chance (often small) of measuring something other than the expected value of a variable in a given measurement. (In field theory, this analogy can be made precise by noting that the the partition functions $$Z = \int D\varphi\ e^{i S[\varphi]/\hbar} \text{ and } Z = \int D\varphi\ e^{-\beta H[\varphi]}$$ for a quantum and a statistical field are simply related by a Wick rotation between real and imaginary time.) Many experts have internalized this analogy so completely that they say things like describing a quantum superposition as "spending most of its time in state $x$ but some of its time in state $y$" as if it were a thermally fluctuating statistical ensemble, even if it's actually a Hamiltonian eigenstate so that strictly speaking nothing actually changes over time at all.

Answer 3

An experimentalist's answer:

Vacuum fluctuation in the absence of real incoming particles interacting with each other is a hypothesis based on the successes of fine structure calculations, with Feynman diagrams, where loops of particle antiparticle pairs are included in higher orders. The calculations succeed in giving accurate numbers for energy levels and higher order effects.

These vacuum loops are connected with vertices to the rest of the virtual particles , and finally to the external incoming and outgoing on mass shell real particles .

Virtual loops are mathematical expressions , the whole calculation obeys energy and momentum conservation.

A virtual loop with no path to external vertices cannot produce real particles, because conservation of energy and momentum in flat spaces ( in the general relativity sense) are absolute, and such a loop in vacuum has zero energy and momentum at each point, but if considered open into on mass e+e- energy conservation would be violated: the loop has zero energy, the real e+e- pair has to have at least the energy of two electron masses.

Lamb shift and spontaneous emission have external vertices so there is no problem, and the loops are energetically legal . The static Casimir effect can be explained by the shielding of the ubiquitous electromagnetic waves by the two close plates as a function of the frequency of the electromagnetic ambience. The dynamical Casimir effect involves external vertices ( what is a mirror if not a plethora of external vertices) and there is no conceptual problem with involving virtual loops. The Hawking effect picks up collective energy from the black hole with a vertex , it is not in vacuum.

In my experimentalist's opinion there is no way that in a complete vacuum a real electron and positron can appear and annihilate due to conservation of energy, a law that is bent only by general relativity considerations. Quantum fluctuations are a mathematical tool, dependent on the specific problem and its boundary conditions.