What's the difference between observable and unobservable objects in a physical theory?

Question

Both can help explain physical effects, but I'm looking for a rigorous definition of “observable” and “unobservable.”

For example, how is the experimental evidence for the existence of particles such as electrons and photons fundamentally different from the evidence for the Casimir effect that virtual particles can explain? We think electrons, quarks, and other particles exist because they provide a good explanation for specific physical effects we've observed. However, we don't observe the particles themselves. Similarly, it seems like virtual particles also help explain and predict specific physical effects, such as the Casimir effect, and I'm not sure I understand where the line is.

To clarify, my question isn't about virtual particles themselves or any specific object; I just used particles vs virtual particles as an example of where I've seen the distinction applied. Instead, my question is about the definitions of “observable” and “unobservable" in the context of physical theories.

Answer 1

The question is really more philosophical than physical. Theories are designed to explain experimental observations, and very few things are available to direct measurement (and even less to human observation without using tools.)

Geocentrism and Heliocentrism
A classical example is the debate about Geocentrism and Heliocentrism - Giordano Bruno and Galileo had it hard for supporting the Heliocentric model. Yet, today one could use either model to describe the solar system - it is just in geocentric model we have to introduce fictitious forces to account for the fact that we are working in a non-inertial reference frame. So the criteria for choosing one or the other have to do with simplicity and computational convenience, rather than with knowing which of the reference frames is "the chosen one" - the criterion is essentially the Occam's razor (although General relativity makes this choice even less ambiguous, see, e.g., Why and when can the Earth be considered an inertial reference frame?)

Microscopic vs. macroscopic/phenomenological models
Another case is developing different theories that describe the same phenomenon, but from different points of view. An example par excellence is the phenomenological thermodynamics, relating measured pressure, temperature and volume, postulating the laws of thermodynamics, including the increase of entropy. The same laws however can be derived in statistical physics, which makes defines (rather than postulates) all these quantities in terms of atoms/molecules constituting the bodies, and some assumptions about the properties of their distributions. Reducing things to more elementary objects and interactions has an intellectual appeal due to the "simplicity", clarify, and generalizability of such a picture, but it is not the only possible view, and it doesn't prove the correctness of such a view. (See in this connection the widespread confusion about different definitions of entropy.)

Phenomenological theories are a lot more widespread than it may seem at first, including:

• Phenomenological electrodynamics (as presented in texts like Jackson or Landau's Electrodynamics of continuous media)
• Hydrodynamics/Fluid mechanics
• Lumped circuit theory (describing circuits in terms of voltages and currents and introducing phenomenological laws like Ohm's law)
• Newtonian mechanics (applied to point-like object, and resorting to forces like friction or normal force.)

Particles and quaziparticles Quantum field theory makes things really ambiguous, since we can apply its logic to excitation in solids, which are very similar to what we normally call particles (fermions and bosons), but which often have bizzarre properties (e.g., unusual dispersion laws.) Are these quasiparticles any less real than electrons, protons, neutrons. Are the latter less real than quarks? Is a Helium nucleus (alpha-particle) less of a particle than a single proton?

Partially the choice is a matter of convenience or tradition, but mostly again an application of the Occam's razor.

Real vs. virtual particles
The difference between particles and quasiparticles is not the same as between real and virtual particles. (Quasi) particles can be observed! Surely, they cannot be observed directly, which means that they might not exist at all, and we could have formulated a theory without them; but, once we put them into the basis of our theory, we can study them and their properties. We can measure their mass, charge, momentum, etc. We cannot do this with a virtual particle - which is a bit of a circular definition, since we call it virtual precisely because it cannot be studied and therefore is not real.

Answer 2

I think that this is an interesting question about Physics. However, to avoid crossing the border between Physics and Philosophy, it is better to put aside terms like exists, or existence. So, let's stick to the question about the meaning of observable and non-observable quantities in Physics. Taking into account the importance of the distinction in quantum mechanics, where observable quantities correspond to self-adjoined operators, or gauge theories, where gauge-dependent quantities are not considered observable, should make clear the importance of a proper definition for Physics.

To the best of my knowledge, finding an explicit and detailed discussion about this central issue in the literature is hard. The only paper I know entirely dedicated to this topic is by Carlo Rovelli, "Partial observables." Physical Review D 65 (2002) 124013.

I'll try to summarize Rovelli's main definitions in the following briefly.

Rovelli starts by saying that

Roughly, observable quantities are the quantities involved in physical measurements. They give us information on the state of a physical system and may be predicted by the theory.

Then, he discusses some weaknesses of such a definition by presenting problematic cases from Quantum Mechanics QM), General Relativity, and Gauge Theories. I just cite the example from QM, where observable, aka measurable quantities, are represented by self-adjoined operators in a Hilbert space. Time is a well-known problematic object since an operator representing it should have a spectrum coinciding with the set of real numbers. Then, its conjugate quantity, the energy should also have an unbounded spectrum. But this is not the case since energy is bounded from below.

According to Rovelli, such difficulties point to the need for distinguishing between what he calls partial and complete observables. His definition of both is the following.

Partial observable: a physical quantity with which we can associate a (measuring) procedure leading to a number.

Complete observable: a quantity whose value can be predicted by the theory (in classical theory); or whose probability distribution can be predicted by the theory (in quantum theory).

He is splitting the initial rough definition into a part related to the measurement and another associated with the theoretical prediction. Even if not stated explicitly by Rovelli, but his examples clarify such a point, his complete observables are made of partial observables. Rovelli must introduce such a distinction to account for situations like those in quantum gravity, where the correlation between two non-observable quantities is observable.

I think this summary could be enough. Interested people could read the full Rovelli's paper. In my opinion, the critical point made by Rovelli is to stress the importance of associating observable quantities with a measurement procedure and a theoretical framework. This combination is important to discuss some of the examples in the original question. Measurable or non-measurable refers to physical properties, not to entities. This observation gets rid of any metaphysical danger about the meaning of existence.

For example, the case of real vs virtual particles from the point of view of Physics is not about their existence but about the possibility of defining physical properties that can predicted and measured.