35

Consider a quantum system described by the Hilbert space $\mathcal{H}$ and consider $A,B\in \mathcal{L}(\mathcal{H},\mathcal{H})$ to be observables. If those observables do not commute there's no simultaneous basis of eigenvectors of each of them. In that case, in general if $|\varphi\rangle$ is eigenvector of $A$ it will not be of $B$.

This leads to the problem of not having a definite value of some quantity in some states.

Now, this is just a mathematical model. It works because it agrees with observations. But it makes me wonder about something. Concerning the Physical quantities associated to $A$ and $B$ (if an example helps consider $A$ to be the position and $B$ the momentum) what is really behind non-commutativity?

Do we have any idea whatsoever about why two observables do not commute? Is there any idea about any underlying reason for that?

Again I know one might say "we don't care about that because the theory agrees with the observation", but I can't really believe there's no underlying reason for some physical quantities be compatible while others are not.

I believe this comes down to the fact that a measurement of a quantity affects the system in some way that interferes with the other quantity, but I don't know how to elaborate on this.

EDIT: I think it's useful to emphasize that I'm not saying that "I can't accept that there exist observables which don't commute". This would enter the rather lengthy discussion about whether nature is deterministic or not, which is not what I'm trying to get here.

My point is: suppose $A_1,A_2,B_1,B_2$ are observables and suppose that $A_1$ and $B_1$ commute while $A_2$ and $B_2$ do not commute. My whole question is: do we know today why the physical quantities $A_1$ and $B_1$ are compatible (can be simultaneously known) and why the quantities $A_2$ and $B_2$ are not?

In other words: accepting that there are incompatible observables, and given a pair of incompatible observables do we know currently, or at least have a guess about why those physical quantities are incompatible?

mpv
  • 5,220
Gold
  • 35,872
  • 7
    Every physical theory is just "a mathematical model". If you are asking about the next higher level theory behind quantum field theory... we don't have that, yet. It may or it may not exist. If you want to find it, then you will have to make a measurement that quantum field theory can not describe. – CuriousOne Feb 29 '16 at 04:16
  • 7
    You say that if observables do not commute, they can have no eigenvectors in common. This is not true. – WillO Feb 29 '16 at 04:19
  • If you like this question you may also enjoy reading this Phys.SE post. – Qmechanic Feb 29 '16 at 18:38
  • 1
    If I tell you that X is the reason for observables not commuting, will you then ask the same question with "observables not commuting" replaced by X? (My point is that either the chain of reasons for things terminates at something that has no reason, or it goes on forever. So either at some point the answer is "No, there is no reason", or these kinds of questions go on forever, so you might want to rethink "I can't really believe there's no underlying reason") – ACuriousMind Mar 01 '16 at 18:41
  • @WillO Are you saying not commute operators can have eigenvectors in common? May I know why? or are you talking about null vector? – Shing Mar 02 '16 at 06:54
  • 2
    @Shing: $\pmatrix{1&1\cr 0&2\cr}$, $\pmatrix{1&1\cr 0&3\cr}$ – WillO Mar 02 '16 at 16:04

7 Answers7

57

Observables don't commute if they can't be simultaneously diagonalized, i.e. if they don't share an eigenvector basis. If you look at this condition the right way, the resulting uncertainty principle becomes very intuitive.

As an example, consider the two-dimensional Hilbert space describing the polarization of a photon moving along the $z$ axis. Its polarization is a vector in the $xy$ plane.

Let $A$ be the operator that determines whether a photon is polarized along the $x$ axis or the $y$ axis, assigning a value of 0 to the former option and 1 to the latter. You can measure $A$ using a simple polarizing filter, and its matrix elements are $$A = \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix}.$$

Now let $B$ be the operator that determines whether a photon is $+$ polarized (i.e. polarized southwest/northeast) or $-$ polarized (polarized southeast/northwest), assigning them values 0 and 1, respectively. Then $$B = \begin{pmatrix} 1/2 & -1/2 \\ -1/2 & 1/2 \end{pmatrix}.$$

The operators $A$ and $B$ don't commute, so they can't be simultaneously diagonalized and thus obey an uncertainty principle. And you can immediately see why from geometry: $A$ and $B$ are picking out different sets of directions. If you had a definite value of $A$, you have to be either $x$ or $y$ polarized. If you had a definite value of $B$, you'd have to be $+$ or $-$ polarized. It's impossible to be both at once.

Or, if you rephrase things in terms of compass directions, the questions "are you going north or east" and "are you going northeast or southeast" do not have simultaneously well-defined answers. This doesn't mean compasses are incorrect, or incomplete, or that observing a compass 'interferes with orientation'. They're just different directions.

Position and momentum are exactly the same way. A position eigenstate is sharply localized, while a momentum eigenstate has infinite spatial extent. Thinking of the Hilbert space as a vector space, they're simply picking out different directions; no vector is an eigenvector of both at once.

knzhou
  • 101,976
  • 1
    this is a truly excellent answer. – JPattarini Feb 29 '16 at 07:58
  • 1
    As @JamesPattarini says, this answer is great. Could you take it one step further and finish the connection to the HUP derivation, maybe the special case of $\Delta x \Delta p$ for simplicity? – user1717828 Feb 29 '16 at 15:23
  • 1
    As a lay enthusiast, I think this is one of the best answers I've seen on PhysicsSE. – Will Feb 29 '16 at 21:38
  • 2
    Thanks! This is my favorite way to explain uncertainty, because it just has so many simplifying properties: the space is finite-dimensional (unlike position and momentum), the photon is spin 1 (so we're dealing with ordinary vectors, not spinors) the photon is massless (so the space only has dimension 2), and it has a nice classical limit (replace the photon polarization with an electric field). I haven't seen this specific example in any textbooks, though. – knzhou Feb 29 '16 at 22:47
  • 1
    @knzhou Actually Dirac relied on the photon example in his Principles Of Quantum Mechanics, see secs.2-4 of chap. 1 in http://www.fulviofrisone.com/attachments/article/447/Principles%20of%20Quantum%20Mechanics%20-%20Dirac.pdf. – udrv Mar 01 '16 at 04:33
  • I'm not sure I like this example. Yes, it neatly explains how the vector space structure allows for non-compatible observables, but the example chosen is a world-vector the components of which are compatible even in the quantum theory. You do not need QM to explain what happens to light when it passes through polarizers. Classically the photon has two observables, the $x$ and $y$ components of polarization and these remain compatible in QM, $x$ and $x+y$ polarization are incompatible observables as you say for geometric reasons. On the other hand the three components of spin are incompatible – Robin Ekman Mar 01 '16 at 12:08
  • for a fundamentally quantum reason and you cannot explain what happens to a polarized beam of electrons in a Stern-Gerlach experiment with classical physics. This, of course, is because photons don't have spin. As you say, they are massless, so they have helicity. Photons live in representations of $U(1)$ which of course is abelian but you want to get at something that comes from a non-abelian structure, so you should use massive particles, which live in $SO(3)$ (or $SU(2)$). – Robin Ekman Mar 01 '16 at 12:09
  • @RobinEkman I think your comments are taking the analogy too far. The whole point of this example is to show how vector space structure allows for incompatible observables, and that's all. There's no reason the analogy has to involve a quantum system. In fact, I'd consider it a bonus that it uses a classical system, because that's easier for people to visualize. You do have a point about the nonabelian structure, but I'd consider that a matter for another question, not a necessary detail to bring up here. – David Z Mar 01 '16 at 12:58
  • @DavidZ But I am not sure what the OP is asking is how the mathematics allow for incompatible ($\Leftrightarrow$ non-commuting) observables. Quote the OP, "Do we have any idea whatsoever about why two observables do not commute? Is there any idea about any underlying reason for that?" The impression you can get from this answer is that it is because our description of the system is badly chosen. This is absolutely false (see: GHZ experiment) and no responsible physicist should say anything that can be construed to support this because then you are not teaching QM but spreading confusion. – Robin Ekman Mar 03 '16 at 15:33
  • This confusion is in the idea you can get from this that it's "really" something classical "under the hood". All the components of the transverse polarization can be simultaneously determined. In fact the experiment described in the answer does this. The intensity after the first polarizer gives you an equation $\cos \theta = x$ with four (or two) solutions corresponding to two (or one) physical situations and the second one picks out one of them, if needed. So this example only shows that non-orthogonal vectors overlap, but it is entirely compatible with that the orthogonal components of the – Robin Ekman Mar 03 '16 at 15:51
  • transverse polarization vector are commuting observables (it has to be, because they are, even in QED). For spin even the orthogonal components don't commute. For spin you really have incompatible observables. Any experiment with photon polarizations will be compatible with that all the transverse components are well-defined. A sequential Stern-Gerlach experiment will not, unless you appeal to hidden variables, but then GHZ and Bell rule that out unless you want to give up locality. – Robin Ekman Mar 03 '16 at 15:57
  • @RobinEkman We may have to attribute this to one of those unresolvable personal differences - my sense was that the answer dispels exactly the impression you get from it. I think the direction analogy does a good job of showing why the uncertainty principle is not a consequence of a badly chosen description, but rather is something more fundamental. – David Z Mar 03 '16 at 16:54
  • @knzhou Late for the party. However, can you explain how you derived A and B. A=|H><H|=((0,0),(0,1)) and B=|d><d|=1/2((1,1),(1,1)), isn't it ? |H> horizontal polarization and |d> represents diagonal. – Chetan Waghela Dec 16 '21 at 19:19
10

Non commuting observables means that a so called measurement is capable of changing the state of the system.

For instance, when there are two observables A and B that fail to commute, there is an eigenvector of A that it is not an eigenvector of B.

When you interact with A then A then B the two results of the A interaction always agree with each other. That means the A interaction always leaves it in a special state, one that gives a specific definite results for an A interaction (the same specific result as the first one gave).

But when that state fails to be an eigenvector of B (and some eigenvector of one fails to be eigen to the other if they fail to commute) then interacting with A then B then A can give two different results for the A interactions.

This proves, definitively, that the interaction with B isn't a passive reveal of preexisting information, but is an interaction that can change the state in question.

Specifically it can change the state from one that gives a particular fixed result for an interaction with A into one capable of giving a different result for an interaction with A.

Timaeus
  • 25,523
  • "This proves, definitively, that the interaction with B isn't a passive reveal of preexisting information, but is an interaction that can change the state in question." /quote I think this is incorrect. One can postulate hidden variables, and it's only through experiments with entangled states (i.e. Bell violations) that we can rule them out. – DanielSank Feb 29 '16 at 05:32
  • @DanielSank Nope. Even hidden variables have it so that a "measurement" of B on an eigenvector of A that isn't eigen to B must change the state into a different state. Hidden variable theories still have states (if nothing else you can say that some things give particular values under A) they just also have hidden variables. The states without the hidden variables are just partial information. But they still are changed. And experiments don't rule out super-determinism and such. So you shouldn't worry about it if an experiment can't rule it out. Focus on what experiments can test. – Timaeus Feb 29 '16 at 05:43
  • 2
    Yes, I think I see your point about super-determinism. Actually the experimental distinction between non-commutation and entangled measurements seems murky in my head now. That usually means I'm about to learn something, so thanks! – DanielSank Feb 29 '16 at 06:07
4

You can make a connection to the fact that the exact physical state of an arbitrary physical system that occupies a finite volume, can be specified with only a finite amount of information. If you consider some observable then the eigenstates may be degenerate, you then need another observable commuting with the first one to lift that degeneracy, if you continue this way you'll eventually end up with a complete set of commuting observables. Since only a finite amount of information is need to specify the state of the system, this means that this set will be finite. It is then guaranteed that you can find observables that don't belong to this set.

In the classical limit, all observables commute. In this limit, the number of distinguishable physical states per unit phase phase tends to infinity.

Count Iblis
  • 10,114
3

Classical mechanics can be expressed in analytical form in terms of position and "conjugate momentum", a term from Lagrangian mechanics. This pair gives us the P, Q variables of Hamiltonian mechanics; when any such pair is quantized you will find that they do not commute. The quantum commutator "inherits" this behavior from the classical Poisson brackets.

So this provides some physical background to your question; the Lagrangian ultimately derives from Newton's Laws of Motion via the Principle of Least Action, a variational principle.

Peter Diehr
  • 7,165
  • 2
    -1: QM is the more fundamental theory. Classical Poisson brackets cannot be an explanation for non-vanishing commutators in QM. It's backwards to even look for such an explanation. What you need to explain is how observables can seem to commute in classical mechanics when fundamentally, in QM, they don't. "Quantization" does not mean taking a limit of a classical mechanical system to get a QM system, it's just a way of making educated guesses. – Robin Ekman Feb 29 '16 at 23:57
  • 1
    @Robin Ekman: you are welcome to explain it in full, including how classical mechanics appears in the limit; however, the usual introduction to quantum mechanics is preceded by a proper exposure to analytical mechanics, and the original ansatz used was indeed that of the conjugate variables and the Poisson brackets. It does, IMHO, provide physical intuition into what is going on. OTOH, I learned QM long ago, and may be out of fashion. C'est le vie! Others have mentioned the theorems of linear algebra that apply. – Peter Diehr Mar 01 '16 at 00:07
  • Yes, that's how you do it pedagogically and the history of it, but it's entirely backwards compared to how reality actually is. Quantization by Poisson bracket -> commutator is a method of generating hypotheses but it can't tell you anything about the peculiarities of QM because there is nothing in classical mechanics that corresponds to non-commuting observables. It's like trying to explain the forces between atoms in a crystal in terms of springs or rubber bands, when you should explain the spring or rubber band in terms of the atoms. – Robin Ekman Mar 01 '16 at 00:17
  • @Robin Ekman: well, I'm primarily an experimentalist, and I've found, over many years, that any source of intuition is useful. I'm perfectly happy using ray optics for one thing, polarization for another, non-linear optics for a third, and quantum entanglement for the next. In fact, that describes the current project that I am working on. And I'm also expected to make it work! – Peter Diehr Mar 01 '16 at 00:22
3

Yes, there is a fundamental reason why some observables do not commute. Some are the non-commuting generators of a group. For example, the angular momenta $J_x,J_y,J_z$ are observables. They are also the generators of rotations in the rotation group. By rotating a pencil with your fingers, you can verify that $Rot_xRot_y$ and $Rot_yRot_x$ do not yield the same pencil orientation. Rotations do not commute and by considering small rotations you deduce the commutation relations $[J_k,J_l]=i\epsilon_{klm}J_m$.

Gary Godfrey
  • 3,354
2

You say that

This leads to the problem of not having a definite value of some quantity in some states.

but what sort of problem is this? It's not a problem of disagreement between theory and experiment. In fact, as I will get to, it's the other way around! If observables commute, we cannot construct theories that agree with experiment. So the problem here is that this is a psychologically or philosophically difficult thing to accept, but that's the way it is and if you don't like it you have to find another universe where the rules are simpler...

Bell's inequalities and GHZ experiments can be argued to show that there really isn't anything behind non-commuting observables. A theory built on only commuting observables simply can't give the correct prediction for the GHZ experiment, but QM does (very easily, too). So non-commuting observables seem to be a very fundamental part of the way our universe works.

Whatever you would suggest is "really" behind quantum mechanics has to make the same prediction as quantum mechanics about the results of a Bell or GHZ experiment, because we have done those experiments and found the result predicted by QM. This rules out that if there is something behind quantum mechanics, all its observables commute.

The universe looks quantum mechanical because it is quantum mechanical.

(There's a loophole in the above: we could allow for faster-than-light signals and then there could be hidden variables. But that's genuinely disturbing to physicists in a sense that non-commuting observables isn't, because if we allow faster-than-light communication we must, according to Einstein, allow conditions in the future to affect what happens in the present. That means I can't trust my experiments anymore because someone from the future might be interfering with them. Non-commuting observables means we have to accept only predicting probabilities in experiments but that's much better than throwing all experiments out because your rival from the future could be sabotaging them.)

Robin Ekman
  • 14,747
2

I'd like to expand a bit on a subtly incorrect statement at the beginning of the original post that was pointed out in the comments:

OP: If those observables do not commute there's no simultaneous basis of eigenvectors of each of them. In that case, in general if $|\phi \rangle$ is eigenvector of $A$ it will not be of $B$. (italics $\rightarrow$ incorrect statement)

Comment (WillO): You say that if observables do not commute, they can have no eigenvectors in common. This is not true.

To give the simplest concrete example, suppose our Hilbert space is finite-dimensional and $A$ and $B$ are non-commuting observables (matrices). Consider these two "expanded" operators on a similarly "expanded" Hilbert space:

$$A'=\begin{pmatrix} A & 0 \\ 0 & 1\end{pmatrix},\,\,B'=\begin{pmatrix} B & 0 \\ 0 & 1\end{pmatrix}$$

Clearly $A'$ and $B'$ also do not commute, but they do share an eigenvector $v=(\,0 \,\,1 \,)^{\text{T}}$ because of the block $1$ in each of their corners.

The moral of the story is that non-commuting observables only imply the OP's first sentence, not the second that I've italicized (see above). Non-commuting observables imply that they can't share an entire common eigenbasis.

Edit

I was wrong about the statement being subtly incorrect (see @Kostya's comments below). The originally intended meaning by the OP (which is how I also first understood it, and which I know was actually the intended meaning by the OP because of their subsequent comments) was incorrect, but the way it was worded actually negated the issue resulting in a technically correct statement. In general an eigenvector of one matrix $A$ will not be an eigenvector of another matrix $B$ when $[A,B]\neq 0$. There may be some eigenvectors of $A$ in common with $B$ (as I originally pointed out), but not all will be in common.

Arturo don Juan
  • 5,321
  • Would you agree that in general for two numbers $a$ and $b$, $a+b$ will not be 5? – Kostya Aug 02 '18 at 15:54
  • @Kostya Um,.. obviously yes. I take the phrase "in general" to be synonymous with "for all/every ...". Sorry I don't follow the implication. – Arturo don Juan Aug 02 '18 at 16:01
  • @Kostya Oh my gosh wow okay that slapped me in the face. So that statement is logically correct then. I'll update my post. – Arturo don Juan Aug 02 '18 at 16:04
  • Yes, its about logical meaning of English "in general". It is not true that for all $a,b: a+b \ne 5$. – Kostya Aug 02 '18 at 16:04
  • @Kostya Still though, at the very least both the OP and myself were confused a bit about that sentence (see the comments to the original post), so I'll leave this up in case anybody else gets confused by it. – Arturo don Juan Aug 02 '18 at 16:07