I have heard that many physical theories (such as quantum gravity) suffer from infinities arising everywhere, which for some reason cannot be regularized or renormalized.
Is there a simple example of such non-renormalizable infinities arising in physics, but with physical content mostly removed, so that only math is shown for simplicity?
Let's consider the mass of the electron. We can write a Hamiltonian (let's just do non-relativistic quantum mechanics for 1 electron, and let's just pretend the spin is zero for simplicity):
\begin{equation} H = \frac{1}{2 m_e^{(b)} }(\vec{p}- e \vec{A})^2 + e \phi(x,t) \end{equation} where $\vec{A}$ is the vector potential and $\phi$ is the scalar potential.
One the one hand, there is a parameter we can put into a Hamiltonian called $m^{(b)}_e$. I will call this the bare mass of the electron, for reasons that will become clear in a minute.
On the other hand, from relativity, the "true" mass of the electron should also include the "self-energy". In other words, since $E=mc^2$, there should be a contribution to the mass of the electron associated with the energy of the electric field that always surrounds the electron, due to its charge.
We can write the total mass of the electron $m_e$ as \begin{equation} m_e = m_e^{(b)} + \frac{U}{c^2} \end{equation} where $U$ is the self-energy. Note that $m_e$ is the real quantity we measure in an experiment (at least, to the level of precision we are working). If you, for instance, applied a magnetic field and watched the electron move in a loop, and used the Larmour precession frequency to infer the mass, the mass you would measure would be $m_e$, not $m_e^{(b)}$.
Now as is well known, the self-energy actually diverges. Let's model the electron as a shell of charge of radius $a$. Then the self energy is
\begin{equation} U = \frac{ke^2}{a} \end{equation} where $k$ is Coulomb's constant. This energy diverges as $a\rightarrow 0$!
Now the modern physical interpretation of this situation, is that we can't really trust this calculation. We have implemented a regulator $a$, and found a divergence if we try to take the parameter to zero. However, we don't really know all of the physics that happens at arbitrarily small distances. Maybe the electron is really some small vibrating loop of string that sets a finite value of $a$. Maybe space is discrete and that prevents the electron from being too small. I'm not advocating for any of these ideas, but the point is, we shouldn't need to know what is happening at arbitrarily short distances, in order to understand how an electron behaves in our lab.
The hypothesis/basic assumption of effective field theory (which is backed up by many calculations and more or less rigorous arguments) even if we don't know all the physics that is happening at arbitrarily small distances, we can parameterize the effect of this physics by local terms in a Hamiltonian. In particular, we can describe the total effect of all the unknown physics, using the parameter $m_e^{(b)}$. We then choose the value of this parameter, so that the results of the theory agree with an experiment. In this case, we need the total mass of the electron, $m_e$, to give a value consistent with experiments.
Therefore, we follow this procedure:
Choose a regulator, $a$, so that the calculations are finite (but depend on an arbitrary parameter $a$).
We choose the value of the bare mass $m_e^{(b)}$, so that the predicted true mass of the electron, $m_e=m_e^{(b)}+\frac{U}{c^2}$, agrees with the observed true mass of the electron. In this case, we can arrange this by setting \begin{equation} m_e^{(b)} = m_e - \frac{ke^2}{a} \end{equation} since then \begin{equation} m_e = m_e^{(b)}+\frac{U}{c^2} = m_e - \frac{ke^2}{a} + \frac{ke^2}{a} = m_e \end{equation} After this procedure, physically observable quantities do not depend on the arbitrary parameter $a$.
For a renormalizable theory, we only need to fix a finite number of parameters using step 2. After that, all parameters are completely fixed and the calculation of any other process is a true prediction of the theory, rather than something used to fix a parameter. For a non-renormalizable theory, in principle we need to fix an infinite number of parameters, so the theory doesn't make any predictions. However, the modern point of view is that we should think of non-renormalizable theories as an effective field theory. What this means is that if we only aim to compute observables to a finite order in a Taylor expansion in terms of the energy of the process we are observing, then we only need to fix a finite number of parameters. This approach has been used very successfully in a number of areas in physics.
If your first reaction to the above process is "that's it?!", that is normal :) On a superficial level it looks like you subtracting out the infinity you get in a calculation by hand and forcing the answer to come out to be what you want. The deep statement is that it is possible to carry out this subtraction in a consistent way, and we don't require an infinite number of arbitrary constants to make actual predictions.
Quantum gravity turns out to be a non-renormalizable theory. This means we can only carry out calculations in an effective field theory sense. In other words, we can only work to a finite order in $E/(M_{\rm pl}c^2)$, where $E$ is the energy of the process under discussion, and $M_{\rm pl}=\sqrt{\hbar c/8\pi G}$ is the (reduced) Planck mass. This is fine so long as $E\ll M_{\rm pl} c^2$, which is true of any conceivable particle physics experiments in the foreseeable future and beyond. What we can't say with certainty at the moment, is what happens when the energies of process exceed the Planck scale. This is what a theory of quantum gravity is supposed to provide.
