What is the meaning of the wavefunction?

Question

I'm taking an introductory quantum mechanics course (at the undergraduate level), for which we're following the Griffiths text. I'm doing well in the course, and can perform calculations fairly reliably, however (especially coming from a mathematical background, where everything must be explicitly justified, explained, and well-grounded) I feel incredibly uneasy about the number of things that seem to be simply given, without much empirical or mathematical backing— first a foremost, Schrödinger's equation and the wavefunction.

The Griffith's text seems to jump right into accepting that our microscopic reality may be described by wavefunctions, which may be interpreted "statistically" according to Born, and which may be solved for according to the Schrödinger equation. I'm familiar with experiments demonstrating the wave/particle duality of fundamental particles (and other small masses), however it seems to me that a mere few paragraphs are insufficient to completely toss out all of our classical models, and accept that reality may be described by wavefunctions, which (for some reason) yield probability distributions when normalized and squared. If the statistical interpretation is merely an "interpretation", then are there alternative ways to understand the wavefunction? What, then, does the raw wavefunction itself actually represent?

I'm aware that Schrödinger's equation cannot be derived from first principles, however there seem to be no rigorous justifications outside of "it seems to work experimentally" (which I suppose is sufficient for an empirical science, but still leaves me feeling rather uneasy).

Where does Schrödinger's equation come from? Why is it that nature is described by wavefunctions, with these very specific properties, and what does the raw wavefunction represent? Are there alternative ways to interpret the wavefunction (that is, other than Born's interpretation)?

Answer 1

Because I can't help myself...

[...] however there seem to be no rigorous justifications outside of "it seems to work experimentally" (which I suppose is sufficient for an empirical science, but still leaves me feeling rather uneasy).

That's not merely sufficient for empirical sciences - that is the only justification a scientific theory could ever have or need. Your unease is understandable, because mathematics and physics are wholly different in this respect. To oversimplify things enormously, mathematicians spend their time exploring the consequences of and connections between rules and frameworks that they build themselves. In contrast, scientists are trying to figure out which rules and frameworks work best to make accurate predictions about the natural world in which we find ourselves.

---

Climbing down off of my soapbox, I'll attempt to answer your question. Consider the following formulation of classical, single-particle Hamiltonian mechanics.

1. The fundamental structure underlying the theory is the phase space $X$. A point $(\mathbf q,\mathbf p)\in X$ corresponds to a possible state of the system by specifying the particle's position and momentum.
2. The physical observables of the system are given by measurable functions from $X\rightarrow \mathbb R$.
3. The possible states $\rho$ of the system correspond to probability measures on $X$. $\rho$ may be a continuous measure with probability density $f_\rho$, or it may be discrete (e.g. a Dirac measure). The former describes a typical state in e.g. statistical mechanics, where we don't know the exact position and momentum of each particle, while the latter is typically what we use in "ordinary" point mechanics.
4. A state $\rho$ evolves in time via a Hamiltonian function $H$, which is a specially chosen observable. Specifically, $$\frac{d}{dt}\rho = -\{\rho,H\}$$ where $\{\cdot,\cdot\}$ is the Poisson bracket, which in canonical coordinates takes the form $$\frac{d}{dt}\rho = -\left(\frac{\partial \rho}{\partial \mathbf q}\cdot \frac{\partial H}{\partial \mathbf p} - \frac{\partial \rho}{\partial \mathbf p} \cdot \frac{\partial H}{\partial \mathbf q}\right)$$

With that out of the way, we can make assign probabilities to the results of measurement outcomes as follows. The probability that an observable $A:X\rightarrow \mathbb R$ takes a value in the set $E\subseteq \mathbb R$ is given by $$\mathrm{Prob}_\rho(A;E) := \rho\bigg(\mathrm{preim}_A(E)\bigg)$$ Assuming that $\rho$ is a continuous measure, this becomes $$\mathrm{Prob}_\rho(A;E) = \int_{\mathrm{preim}_A(E)} f_\rho(\mathbf r,\mathbf p) \mathrm d\mathbf r \mathrm d\mathbf p$$ If $\rho$ is a Dirac measure supported at the point $(\mathbf q_0, \mathbf p_0)$, then $$\mathrm{Prob}_\rho(A;E) = \begin{cases} 1 & A (\mathbf q_0,\mathbf p_0)\in E \\ 0 & \text{else}\end{cases}$$

---

That brisk summary contains the two primary ingredients necessary for a physical theory:

1. Given a state, we need to be able to compute the probabilities of various measurement outcomes, and
2. We need to be able to evolve states forward in time

It turns out that this framework is incredibly successful, but it has some limitations which render it fundamentally incapable of accurately modeling certain phenomena. Instead, we must turn to quantum mechanics, which differs from classical physics in one fundamental way - the existence of incompatible observables.

Two observables $A$ and $B$ are called compatible if we can define their joint probability distribution - that is, if we can meaningfully assign probabilities to simultaneous measurements of both. In classical mechanics, this is can always be done; given a state $\rho$, the probability of measuring observables $A$ and $B$ to take values in the sets $E_A$ and $E_B$ is simply $$\mathrm{prob}_\rho(A,B;E_A,E_B) = \rho\bigg(\mathrm{preim}_A(E_A)\cap \mathrm{preim}_B(E_B)\bigg)$$

The incompatibility of certain quantum observables means that we require a more general logical framework which allows for the possibility that two observables simply don't have a well-defined joint probability distribution.

---

I'll now describe such a framework. Here's a formulation of quantum mechanics:

1. The fundamental structure underlying the theory is the Hilbert space $\mathscr H$ with inner product $\langle \cdot,\cdot\rangle$.
2. The physical observables of the system correspond to the self-adjoint operators $A:\mathscr H\rightarrow \mathscr H$.
3. The possible states $\rho$ of the system correspond to density operators - that is, positive self-adjoint operators with trace $1$. Given a unit vector $\psi\in \mathscr H$, one possible form for a density operator is $\rho_\psi: \phi \mapsto \psi \langle \psi, \phi\rangle$ (in bra-ket notation, we woudl write $\rho_\psi = |\psi\rangle\langle \psi|$). If the state has this form it is called pure, but not all states do.
4. A state $\rho$ evolves in time via a Hamiltonian operator $H$, which is a specially chosen observable. Specifically, $$\frac{d}{dt}\rho = -[\rho,H]/i\hbar$$ where $[\cdot,\cdot]$ is the commutator bracket for two operators on $\mathscr H$.

With that out of the way, we can make assign probabilities to the results of measurement outcomes as follows. Any self-adjoint operator $A:\mathscr H \rightarrow \mathscr H$ corresponds to a unique projection-valued measure$^\ddagger$ $\mu_A$, which maps a set $E\subseteq \mathbb R$ to an orthogonal projection operator $\mu_A(E)$. Given a state $\rho$, the probability that $A$ takes its value in $\mu_E$ is given by

$$\mathrm{prob}_\rho(A;E) = \mathrm{Tr}\bigg(\rho \mu_A(E)\bigg)$$

If $\rho$ is a pure state, then the probability becomes $$\mathrm{prob}_{\rho_\psi}(A;E) = \mathrm{Tr}\bigg(\rho_\psi \mu_A(E)\bigg) = \langle \psi, \mu_A(E)\psi\rangle$$

For a specific example, if $\mathscr H=L^2(\mathbb R)$ with the inner product $\langle \psi,\phi\rangle = \int \overline{\psi(x)} \phi(x) \mathrm dx$, and $A=\hat X$ is the position operator $\big(\hat X\psi\big)(x) = x \psi(x)$, then $\mu_{\hat X}(E) = \chi_E$ where $$\chi_E(x) = \begin{cases}1 & x\in E\\ 0 & \text{else}\end{cases}$$

Therefore, we have$^{\ddagger\ddagger}$

$$\mathrm{prob}_{\rho_\psi}(\hat X;E) = \int_\mathbb R \chi_E(x)|\psi(x)|^2 \ \mathrm dx = \int_E |\psi(x)|^2 \mathrm dx$$

---

Once again, we have our necessary ingredients for producing probabilities and evolving states. But now we have a crucial difference, because we have incompatible observables. If $A$ and $B$ are self-adjoint operators with projection-valued measures $\mu_A$ and $\mu_B$ respectively, then the probability of measuring them to take values in the sets $E_A$ and $E_B$ individually are found via the orthogonal projection operators $\mu_A(E_A)$ and $\mu_B(E_B)$.

We might imagine that measuring them together would correspond to using the operator $\mu_{A,B}(E_A,E_B) = \mu_A(E_A)\mu_B(E_B)$, and we would be correct provided that $A$ and $B$ (and by extension, $\mu_A(E_A)$ and $\mu_B(E_B)$) commute. If they don't, then $\mu_A(E_A)\mu_B(E_B) \neq \mu_B(E_B)\mu_A(E_A)$ and the order in which they are measured matters. We say that $A$ and $B$ are incompatible observables.

The existence of incompatible observables has additional implications. For example, a state $\rho$ is called sharp if, for every orthogonal projection operator $P$, $\rho(P)=0$ or $\rho(P)=1$ with no intermediate values (the Dirac measure states from classical physics are sharp). The existence of incompatible observables implies that no sharp states can exist.

---

Of course, you may ask why such a framework is the right one. There are some reasons that it is not quite as arbitrary as it seems.

1. Gleason's theorem tells us that given some reasonable assumptions, any probability measure which assigns probabilities to orthogonal projectors can be manifested as a density operator.
2. The Gelfand-Naimark theorem tells us that any $C^*$ algebra of observable quantities can be realized as bounded operators on some Hilbert space. By implication, we need only motivate the fact that our observables should have the structure of a $C^*$ algebra to completely justify our use of a Hilbert space as a concrete vehicle for the theory.
3. Classical observables have the structure of a Poisson algebra, which differs from a $C^*$ algebra. But if we jump from the former to the latter, then we are led immediately to the standard formulation of quantum mechanics. I don't know how to motivate that, really, but it's worth considering.

---

$^\ddagger$This is why we require that observables are self-adjoint.

$^{\ddagger\ddagger}$This is why $|\psi(x)|^2$ can be identified with the probability density associated with the position observable.

Answer 2

The reason for the paucity of what your textbook tells you is that the question of what these things "mean" beyond just being formal devices we can use to extract predictions for how a particular experiment will behave are not widely-agreed upon by the physics community, even 100 years after quantum mechanics has been formulated. There's also an unfortunate, though I hope now it isn't quite as lively, strand of "philosophy" that goes under this by the name of "shut up and calculate" (though my preferred version when PO'd at this sentiment includes another word there not suitable for this forum). Basically, give up on trying to ask what kind of "thing" quantum theory is really trying to describe the world in terms of, just use it to run experiments and publish fast so you don't perish.

If I were going to answer you I'd suggest that it's kinda roundabout to see it from the Hilbert formalism, but the quantum theory is an informationally-based theory. In fact, you don't really use "wave functions" but, as another answer here points out, density operators. It's kind of an unfortunate matter that the commonly-used Hilbert formalism has two things in it that "look like" states, respectively the vectors and then the density operators, but it's more a peculiar artifact, similar to a gauge theory, of that formalism.

A quantum state represents your (or generally, the "agent"'s) knowledge, or possessed information, about the physical parameters of a system. There is a state $\hat{\rho}$ corresponding to having no information at all - it is the identity density operator. There exists a density operator corresponding to having zero information. There are also operators that have maximal information: when you have the greatest information you can have regarding all parameters of the system, you have what is called a "pure state", and there exists a Hilbert space on which those density operators project down to one-dimensional subspaces of that space, which then can in turn be represented by a single vector drawn from it. Those vectors are where you get your wave functions. When you gain new information, you revise your density operator accordingly, just like revising a classical probability distribution.

If this seems odd because there are two "layers" going on here, one with vectors and one with operators, yes it is - and in fact, it isn't necessary. The other post mentions the C*-algebra formalism, which elides this redundancy - in fact, you can actually construct many different Hilbert spaces that are suitable to represent the same system of quantum states provided you don't add further operators to your observable set. The algebra and operators matter more than the vectors and it's kind of an accident of history that the formalism is in the state it is.

The thing that makes quantum theory so odd is that when you reach a state of maximal informedness (a pure state), you can actually keep extracting information from the quantum system, but your gains in information about one set of aspects of that system are balanced by losses of information about other aspects. And it's not in the sense that you're "forgetting" or that something reaches into your brain to wipe it out from there, it's that the prior information becomes physically invalid in that if, after retrieving the new information from the system you go back and try to look at a different property you accessed before, you can experimentally see that what comes up doesn't jive with prior records. In that sense, it's like that the Universe keeps a maximum tally of information that it's going to use to fill in the properties of the system and when you want to get more to stake out one property it has to bail out some from another property in order to "make some room". Under the projection postulate, your vector "jumps" around randomly in the Hilbert space as you try to update it further and further.

However, many dispute these ideas. It seems unsatisfying to many that our best theory would have to be one that inherently describes things only from the point of view of what knowledge we have about them, and ideally we'd like to say how the Universe "really works" behind the scenes. Hence some suggest that maybe what's really going on is that the vectors in Hilbert space are literal, and different from the density operators that form knowledge. This gives the Copenhagenism and many-worlds interpretations. Some suggest that the theory is only about knowledge and has no physical content at all; this is "quantum Bayesianism". Some even try to come up with ideas involving stuff travelling backward in time (so-called "transactional interpretation"). And so forth. Many if not all of these ideas can "implement" quantum theory, but we'd have no way to know if any of them are actually true unless we discover something experimentally which quantum theory cannot account for but perhaps a suitably-modified version of one of them can. To date, though, quantum theory has proved absolutely bulletproof - even novel physics our evidence only hints at, like dark matter or neutrino mass, that is far from fully understood, provides no indication of a fundamental inadequacy in quantum theory.

Insofar as making your title question, wave functions or better Hilbert space vectors, are really best seen as a mathematical "substrate" for operators to act upon. What's physically relevant is the probability distributions generated by them.

Answer 3

Your questions are all good ones that you should naturally be asking yourself. They are not the kind of questions for which there are really good, crisp answers. Our current understanding of quantum physics is that experts who care about these foundational issues peer at it from a bunch of different angles, poke it with a stick, and try out different mental frameworks. So I'll just try to say a few things about what these views-from-different-angles look like.

Many modern descriptions of quantum mechanics look more like descriptions of quantum information. Examples:

Aaronson, "Is Quantum Mechanics An Island In Theoryspace?," http://arxiv.org/abs/quant-ph/0401062

Masanes and Mueller, "A derivation of quantum theory from physical requirements," https://arxiv.org/abs/1004.1483

Dakic and Brukner, "Quantum Theory and Beyond: Is Entanglement Special?," https://arxiv.org/abs/0911.0695

Kapustin, https://arxiv.org/abs/1303.6917

Seen from this angle, the fundamental building blocks are qubits. Qubits are described by wavefunctions, but unlike the wavefunctions you've probably been seeing in your class, these functions have a domain that is discrete, like {0,1}. Quantum computers can handle problems, such as efficiently factoring large numbers, that classical computers can't do. This is fundamentally because classical information can be converted losslessly to quantum information, but not vice versa. This "lossiness" is the same kind of lossiness you experience when you give up talking about wavefunctions and talk only about probabilities.

You express some discomfort: "I'm familiar with experiments demonstrating the wave/particle duality of fundamental particles (and other small masses), however it seems to me that a mere few paragraphs are insufficient to completely toss out all of our classical models, and accept that reality may be described by wavefunctions, which (for some reason) yield probability distributions when normalized and squared."

Here are a couple of ways to see that quantum physics can't be tamed or confined to a little corner, where it doesn't threaten classical physics.

You mention small masses. Actually there is no mass or distance scale that separates quantum physics from classical physics. The only parameter with units that is relevant is Planck's constant, and you can't manipulate Planck's constant to get units of mass or distance alone. This makes it sort of inevitable that if you accept quantum physics for electrons and photons, you have to accept it for people and planets.

Another way to see that quantum physics spreads inevitably is to think about what it is in the photoelectric effect that prevents the same photon from being absorbed by two different atoms, thereby violating conservation of energy, since energy E goes away, and energy 2E gets created. Because the wavelengths of visible and UV light are so big compared to atoms, a single photon will normally impinge on thousands or millions of atoms. One way of dealing with this that was originally proposed was to give up on conservation of energy as a perfect conservation law, and instead make it only hold at a statistical level. However, experiments by Geiger and Bothe proved that this was not the case. The only resolution we can find is that there have to be nonclassical correlations between the different atoms -- if atom A receives the photon, then there is an (anti-)correlation that says B didn't absorb it. This is a specific example of a more general fact, which is that quantum theories don't play nicely with classical ones. It's not possible to have, for example, quantum-mechanical photons that interact with classical atoms. Because of this, proving that one system acts according to quantum mechanics is sufficient to prove that all other systems do, if they can interact with the original system.

Continuing with the photoelectric effect, think about that relation E=hf between the energy of a photon and its frequency. That's really all the Schrodinger equation is. (If your book talks about "the" Schrodinger equation as being the thing with $\partial^2\Psi/\partial x^2$ in it, then that's really not quite right. That's just one special case of the Schrodinger equation.)

What, then, does the raw wavefunction itself [as opposed to probabilities] actually represent?

Concretely, the wavefunction includes phase information, which the probabilities don't tell us. The fact that we can do double-slit interference with electrons tells us that these phases matter. However, it's important to realize that only relative phases are definable. For example, if I take the wavefunction of every electron in your body and flip its sign, you will feel absolutely no effect.

This is similar to the way electrical potentials work. You can define anything you like as your ground. Only differences in voltage are meaningful. So in that sense the electric potential is fake. However, in a lot of calculations it's helpful to make some arbitrary choice of the additive constant and work with some symbol for the potential. Same with the wavefunction.

Answer 4

Are there alternative ways to interpret the wavefunction (that is, other than Born's interpretation)?

So there are indeed such alternative ways. I will not try to describe all of them, just one direction I am interested in.

So, given a wave function, you know how to costruct a (4-)current. According to Born's interpretation, it is a current of probability. For example, the zeroth component of the current is the probability density.

However, Schrödinger initially interpreted this component as the charge density (here I ignore some constant factors, such as the elementary charge). This interpretation has its difficulties, as, for example, wave function typically spreads in space with time. This is in tension with the idea of discrete particles.

I proposed a modification of such interpretation where this component is only a smoothed (coarse-grained) charge density approximating a discrete charge density of a large collection of point-like particles and antiparticles, such that a great number of "first" Fourier components coincide for the smooth function and the discrete charge distribution. Such plasma-like description seems immune to the problem of spreading of the wave function.

Answer 5

This is maybe a little... heterodox but section 4.5 of the book Waves in Fluids by James Lighthill gives some motivation for the Schrödinger equation. I found it really illuminating.

The thinking goes like this. For any constant-coefficient wave equation in any medium, you can learn a lot by deriving what the dispersion relation is -- for any solution

$$\phi = \exp(i(\mathbf{k}\cdot\mathbf{x} - \omega t)),$$

what is $\omega$ as a function of $\mathbf{k}$? For the regular scalar wave equation, there are two branches $\omega = \pm c|\mathbf k|$. But there are more exotic dispersion relations for, say, water waves in a rotating tank. One of the really important consequences is that energy tends to propagate according to the group velocity of the wave, i.e. along paths

$$\dot{\mathbf{x}} = \frac{\partial\omega}{\partial\mathbf k}.$$

What if the medium is no longer homogeneous -- for example, the wave speed depends on position? At first, you might just throw your hands up and say that Fourier analysis is no longer useful in helping us write down exact solutions. But this isn't really true as long as the wavelength of the disturbances you're interested in is much smaller than the characteristic length scale for the inhomogeneities of the medium. In that case, you can use the geometric "optics" approximation, and I put "optics" in quotes because it still works for acoustic, seismic, or water waves and not just light. In that case, we let $\omega$ be a function of both wavenumber and position. Now energy still tends to propagate in space according to the group velocity, but now the wavenumber can change along ray paths. How does the wavenumber change? By God, it's a Hamiltonian system:

$$\begin{align} \dot{\mathbf{x}} & = +\frac{\partial\omega}{\partial\mathbf k} \\ \dot{\mathbf{k}} & = -\frac{\partial\omega}{\partial\mathbf x} \end{align}$$

So waves of sufficiently high wavenumber can behave almost like particles. I want to emphasize that this gets used in lots of different fields. Want to see what path the energy takes through the interior of the earth when a quake goes off? Well, we have a rough idea of the density variations, and we know how density goes into the seismic wave equation...

You can "derive" the Schrödinger equation by asking this question in reverse: What PDE gives the Hamiltonian equations of motion as its geometric optics limit? Now you might find all this a little unsatisfactory because it still doesn't tell you what the wave function means or that its square modulus is a probability density. But it at least helps to give more of a clue of where the Schrödinger equation comes from rather than just pulling it out of a hat.

Answer 6

Schrodinger's equation comes from wave equation,

$\frac{\partial^2 \Psi}{\partial x^2}=\frac{1}{v^2}\frac{\partial^2 \Psi}{\partial t^2}\tag*{}$

whose solution is,

$\Psi(x,t)=A\ e^{\frac{\iota}{\hbar}(E\ t-\mathbf p\cdot \mathbf x)}$

where using relation, $E=\hbar \omega$ from Planck's distribution law, and $\mathbf p=\hbar \mathbf k$ from De-broglie's hypothesis for matter wave.

Also from classical mechanics, total energy is sum of kinetic and potential energy, so $E=\frac{p^2}{2m}+V$, where $V$ is potential function depends upon $x$ and $t$.

$\frac{\hbar}{\iota}\frac{\partial \Psi}{\partial t}=E\Psi \implies \hat H\rightarrow \frac{\hbar}{\iota}\frac{\partial}{\partial t}$ similarly for,

$\hat p\rightarrow \iota \hbar \frac{\partial}{\partial x}$ are energy and momentum operators.

In quantum mechanics, there are operators for observables, that operate on state or wavefunction giving scalar observable. Thus using classical relation of total energy in terms of operators gives,

$\frac{\hbar}{\iota}\frac{\partial \Psi}{\partial t}=-\frac{\hbar^2}{2m}\frac{\partial^2 \Psi}{\partial x^2}+V\Psi\tag*{}$

When Planck gave that interaction of energy (energy is not definite for a particle like mass and volume) is not continuous but discrete or quantized to minimum limit, this opened way for particle nature of wave as wave is effect. Later De-broglie proposed that matter also have wave or represented as wave, this lead to non-localization of particle or understanding of particle at smaller level.

The wavefunction represents probability amplitude whose square is probability and to find average value of an observable it's operated by operator and correlated with original wavefunction gives probability of an observable. The wavefunction is stationary and operator is dynamic. Mere observables given in terms of probability doesn't make quantum mechanics different but the mathematics used to describe it, matrix mechanics.

The interpretation came from mathematics lead to Heisenberg's indeterminacy result and which is result of non-commutative matrix multiplication or wave representation by Schrodinger. Fourier transform that relate one domain frequency with other domain time shows that how change in one variable changes other variable, this is indeterminacy in observables when one is known before.

The indeterminacy principle means that present comes from past and if one wants to observe present state, then at smaller level that destroy or change future, as observation changes state or wavefunction's phase which is scalar value of observable and amplitude also. This is contrary to deterministic classical mechanics in which by knowing present, future or trajectory can be known. So trajectories or how or why become less important then what and when in quantum mechanics unless seen in light of classical.

Answer 7

Griffiths is adopting an educational method where you learn the physical meaning of the mathematical tools by applying the tools and seeing where they lead.

Schrodinger's equation is all about energy, and the idea that there is a relationship between energy and development over time. That's not so far from more familiar classical ideas. Other answers have helpfully spelled this out, so I won't repeat that. You can write it as $\hat{H} |\psi\rangle = i \hbar \frac{d}{dt}|\psi\rangle$ with $\hat{H} = \hat{p}^2/2m + \hat{V}$ and thus postpone the question of the relationship between the operators, kets, and physical things. The behaviour of the operators is mostly captured by introducing $[\hat{x},\hat{p}] = i \hbar$.

Coming to wavefunctions and so on, quantum mechanics invites us to frame our questions in observable terms. The answer to the question "is the wavefunction a physically real wave of some kind?" is "tell me what experimental evidence would constitute evidence of physical reality, and I (the set of mathematical tools) will predict whether that evidence will be found." If the notion of physical reality has built into it a notion of locality (i.e. all effects are in the timelike future of their causes), then quantum mechanics will tell you that there will be evidence against that combination if you apply it directly to that which determines the probability distribution of correlations. This evidence comes in the form of experimental Bell inequality violations.

In practice I think we mostly come to the conclusion that there is no great value in trying to assert a kind of direct physical reality of wavefunctions. They are regarded rather as useful mathematical tools. Sense can be made of this by asserting that what really happens in the universe is a sequence of irreversible probabilistic events, and quantum theory furnishes a mathematical apparatus which can be used to calculate the probability distribution for a set of future events, given a set of past events. The apparatus concerns quantum fields, and we can assert that quantum fields are physically real, but this is not the same as asserting that either states in Hilbert space or wavefunctions are physically real.

Answer 8

A wavefunction is the description of the quantum mechanical ensemble, similar to how a probability distribution is the description of a statistical ensemble. What a wave function is not is the description of a single system. All by itself a wave function also does not connect to realty. Additionally to the wave function we also have to select a preparation and a measurement operator and then we can make statements about the probabilities of events in many copies of a quantum system. What we can never do is to predict the occurrence of a single event.

The Schroedinger equation came, at least according to Feynman, from Schroedinger's mind. How he got there is described in a series of five papers of his. You can find them online, I believe. I tried to read them, but to be honest, much of the reasoning in them is very opaque, at least to me. That's probably because I am not Schroedinger. It is also no more important than how Newton arrived at Newtonian mechanics. He simply did and we can test that he was correct about a great many phenomena with it.

The more important question is this: is the Schroedinger equation even the correct equation? The answer to that is a resounding "No!". The Schroedinger equation is a very limited approximation and mostly unphysical. The "correct" equations are those of quantum field theory (at least when we reduce it to practice in the standard model). What the Schroedinger equation can do for you is to give an easily understandable toy model for the real thing, but if you want to understand the actual reality of quantum mechanical systems, then you will have to go beyond.