Is there a physical example of a distribution that has a diverging standard deviation (like the Cauchy distribution) and is there an intuitive reason for the standard deviation diverging?
Is there a physical context where I should expect my standard deviation to be not well-defined?
The standard deviation is one example of a metric to characterize the width probability distributions. There is no particular physical meaning to the standard deviation diverging, in general; you just need to use a different measure of the width of the distribution. It is just a mathematical tool, that is useful in some contexts, and not others.
The Cauchy distribution is also called a Lorentzian distribution in physics, and appears very frequently in characterizing modes of a system with dissipation in classical mechanics, or unstable particles that decay (also called resonances) in quantum mechanics. It arises naturally when you solve the equation for a damped, driven harmonic oscillator in Fourier space (compare Eq 120 in the notes I linked to with the first equation on wikipedia). The fact that the standard deviation of the Lorentzian diverges poses no problem to using this distribution in those contexts. Often, the width of a Lorentzian is characterized in terms of the full width at half maximum (FWHM).
