How is the "normalisation" of non-normalizable states chosen?

Question

This question is about non-normalisable states in quantum mechanics, e.g. the eigenstates of the position operator $|x\rangle$ which are defined by the eigenvalue equation

$$\tag{1} \hat{X}|x\rangle = x|x\rangle.$$

Since the spectrum of the operator is continuous, the states are not square normalisable. Instead one has an equation such as

$$\tag{2} \langle x'|x\rangle = \delta(x-x')$$

which gives the overlap meaning as a distribution. However it is clear that there is still at least an ambiguity in choosing a constant, i.e. why no have

$$\langle x'|x\rangle = a \times \delta(x-x')$$

where $a$ is $a \in \mathbb{R} $.

---

So I went to see how Equation (2) is derived. In Schleich's textbook on quantum optics in phase space I found the following, very short and in my opinion not clear/complete derivation.

From Equation (1) which can be taken to be the definition of the position operator and the assumption that the operator is hermitian which is a fundamental postulate of quantum mechanics one obtains:

$$\tag{3} \left( x - x' \right) \langle x'|x\rangle = 0.$$

That was easy so far, but then Schleich jumps immediately to equation (2). But the solutions for $\langle x'|x\rangle$ of (3) are all functions/generalised functions that are non-zero everywhere except at $x=x'$. So my question is: Why is (2) the unique solution? Where does the additional information come from?

My suspicion is that we impose another requirement, which is probably the completeness of the states, which would make (2) clear since it is the identity element in the position representation.

Answer 1

Eq. (2) is not uniquely determined by saying that the $\lvert x\rangle$ are "position eigenstates", which is bad terminology to begin with since they are not, in fact, admissible states, since states by definition belong to the Hilbert space and must be normalizable. Their proper treatment requires the notion of rigged Hilbert spaces.

However, $\delta(x-x')$ is simply the condition for the "basis" $\lvert x\rangle$ to be orthonormal, i.e. eq. (2) is imposed rather than derived from anything. Note that that form of the "inner product" implies $$ 1 = \int \lvert x\rangle\langle x\rvert\mathrm{d}x, $$ so the $\lvert x\rangle$ resolve the identity in a generalization of the way a usual countable orthonomal basis does as $1 = \sum_n \lvert\psi_n\rangle\langle\psi_n\rvert$. So indeed, it is completeness and orthonormality that yield eq. (2).

Answer 2

Using Sakurai's Modern Quantum Mechanics as a basis for this answer, before looking at this in a continuous system, it might be helpful to look at it in a discrete system first:

For eigenstates of some operator, $\hat{A}$, $\mid a'\rangle$ and $\mid a''\rangle$, such that $\hat{A}\mid a'\rangle = a'\mid a'\rangle$ and $\hat{A}\mid a''\rangle = a''\mid a'\rangle$ gives the equation $\left(a' - a''\right)\left\langle a''\mid a'\right\rangle = 0$

So, we want the eigenbasis to be orthonormal, i.e. $\left\langle a'\mid a'\right\rangle = 1$ and $\left\langle a''\mid a'\right\rangle = 0$ for $a'' \neq a'$. Helpfully, this is also the definition of the Kronecker delta: $\left\langle a''\mid a'\right\rangle = \delta_{a', a''}$

Assuming that the whole space is spanned by this eigenbasis, we now have a complete basis.

Now, extending this to a continuous system, the Dirac delta function is the continuous version of the Kronecker delta - see e.g this question, so it's entirely logical and intuitive to replace the Kronecker delta in discrete systems with the Dirac delta function. Yes, using the Kronecker delta in the first place is really just a convention that gives completeness of states (as you've said), but if it ain't broke...

Now, in a continuous system, we have that the identity is $1 = \int dx\, \mid x\rangle\langle x\mid$, giving $\left\langle x'\mid x''\right\rangle = \int dx\, \left\langle x'\mid x\rangle\langle x\mid x''\right\rangle$ and so the Dirac delta function matches this.