Canonical commutation relation $[X,P]=i\hbar I $ in the eigenbasis of harmonic oscillator

Question

I came across this problem when studying the quantum harmonic oscillator. I want to calculate the commutator of position and momentum operators represented by $4\times 4$ matrices in basis $n,$ i.e. the basis of the harmonic oscillator eigenstates. It is stated in shankar p.208 that in the $n$ basis X and P are represented by the matrices $$X= \sqrt{\hbar/2m\omega}\begin{bmatrix} 0 & 1 & 0 & 0\\ 1 & 0 & \sqrt2 & 0 \\ 0 & \sqrt2 & 0 & \sqrt3 \\ 0 & 0 & \sqrt3 & 0 \end{bmatrix} $$ and $$ P= i\sqrt{\hbar m\omega/2}\begin{bmatrix} 0 & -1 & 0 & 0\\ 1 & 0 & -\sqrt2 & 0 \\ 0 & \sqrt2 & 0 & -\sqrt3 \\ 0 & 0 & \sqrt3 & 0 \end{bmatrix}. $$

Calculating the commutator I obtain $$ [X,P] = i\hbar \begin{bmatrix} 1 & 0 & 0 & 0\\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & -3 \end{bmatrix}. $$

Shouldn't the relation $[X,P]=i\hbar I $ hold true in every basis? Why am I not getting the correct commutator in this case?

Answer 1

$[X,\,P]=i\hbar I$ with $n$-dimensional $X,\,P$ for finite $n\ne0$ gives the contradiction$$0=\operatorname{tr}[X,\,P]=in\hbar,$$so this commutator is unachievable in finite dimensions. Shankar shows you how the infinite-dimensional matrices start, in their top left corners.

Answer 2

Shouldn't the relation $[X,P]=i\hbar I $ hold true in every basis?

That's the thing $-$ this isn't a basis! Instead, it is a restricted subspace of finite dimensions. And, as mentioned in the existing answer, the canonical commutation relations cannot hold in finite dimensions.

This is because, in finite dimensions, the trace of any commutator is always zero: \begin{align} \mathrm{Tr}\mathopen{}\left([X,P]\right)\mathclose{} & = \mathrm{Tr}\mathopen{}\left(XP-PX\right)\mathclose{} \\ & = \mathrm{Tr}\mathopen{}\left(XP\right)\mathclose{} - \mathrm{Tr}\mathopen{}\left(PX\right)\mathclose{} \\ & = 0, \end{align} because the trace is cyclic and $\mathrm{Tr}(AB)=\mathrm{Tr}(BA)$ wherever both are defined. For the full-blown position and momentum operators on the infinite-dimensional Hilbert space $L_2(\mathbb R)$, this is not a contradiction, because neither of $XP$ and $PX$ are trace-class, so neither of their traces is independently defined.

The space you're working in is known as the truncated harmonic oscillator, and it's quite a different beast to the full-blown infinite-dimensional version. In particular, if you restrict it to $N+1$ dimensions (so the space is spanned by $\{|0⟩,|1⟩,\ldots|N⟩\}$), the position-momentum commutator comes out to $$ [X,P] = i\hbar \bigg[I-(N+1)|N⟩⟨N|\bigg], $$ which is as close as you can get to the ideal thing. When you work in this space, the idea is always to make $N$ big enouch that the projector at the end there, $|N⟩⟨N|$, stops having any overlap with any states that interest you, which then implies that the action of $[X,P]$ coincides with that of $i\hbar I$ for that subset of states.

As a shameless plug, my first paper, "On the spectrum of field quadratures for a finite number of photons" [J. Phys. A: Math. Theor. 45, 395303 (2012), arXiv:1109.5724] (where "field quadratures" means $X$ and $P$ in your notation), discusses this space in detail; you might find it interesting as an example of how to work there.

---

So much for the math as you have written it. You also attribute those equations to Shankar, and that is incorrect. What Shankar writes is this:

Notice the ellipsis ($\cdots$ and $\vdots$) at the edge of the matrix, indicating that this is an infinite matrix which keeps on going indefinitely. Truncating this matrix, as you should now see, substantially changes the game.