Here is a hopefully more conceptual perspective from [1] and [2] on why one chooses e.g. $\hat{a}_p u e^{-ipx}$ rather than $\hat{a}_{\mathbf{p}} \overline{u}_{\mathbf{p}} e^{+ipx}$ or something to represent the annihilation of a particle in the wave operator expansions.
In non-relativistic quantum mechanics, if
$$\psi_n(x,t) = \psi_n(x) e^{- i E_n t}$$
are the stationary states of the $\hbar = 1$ Schrodinger equation, and $\hat{A}$ is some operator, then the matrix element of $\hat{A}$ for a transition from $|i>$ to $|f>$ is given by
$$A_{fi}(t) = \int \psi_f^*(x) \hat{A} \psi_i(x) dx e^{-i(E_i - E_f)t} = A_{fi} e^{- i \omega_{if}t}$$
where we set
$$\omega_{if} = E_i - E_f.$$
If we then do second quantization for a system of identical particles such that the stationary states of a single such particle are the above stationary states, and assume $E_i, E_f,E_k$ are all non-negative (important for comparison with the discussion below), then a transition where the number of particles in the $k$'th stationary state increases by one has a final energy
$$E_f = E_i + E_k$$
so that $\omega_{if} = E_i - E_f = - E_k$ implies the time dependence of the matrix element of this transition is
$$A_{fi}(t) = A_{fi} e^{+i E_k t},$$
Thus single-particle operators with a $e^{+i E_k t}$ time dependence represent transitions where the particle number increases by one (creation operators). Similarly if
$$E_f = E_i - E_k$$
is the energy of a transition where the particle number in the $k$'th stationary state decreased by one we have $\omega_{if} = E_i - E_f = + E_k$ so that the time dependence of the matrix element is
$$A_{fi}(t) = A_{fi} e^{-i E_k t}.$$
so single-particle operators with a $e^{- i E_k t}$ time dependence represent transitions where the particle number decreases by one (annihilation operators).
Since in second quantization of a system of identical particles one promotes the general expansion of a single-particle wave function in terms of it's stationary states $\psi(x,t) = \sum_n a_n \psi_n(x) e^{-i E_n t}$ to a quantum field operator
$$\hat{\psi}(x,t) = \sum_n \hat{a}_n \psi_n(x) e^{-i E_n t}$$
we can interpret a non-relativistic quantum field operator as a sum of single-particle annihilation operators $\hat{a}_n$ where $\hat{a}_n$ is an operator annihilates a particle in the $n$'th stationary state, and their adjoint
$$\hat{\psi}^{\dagger}(x,t) = \sum_n \hat{a}_n^{\dagger} \psi_n^* e^{+i E_n t}$$
as a sum of single-particle creation operators, where $\hat{a}_n^{\dagger}$ creates a particle in the $n$'th stationary state. We can guess based off the above what the 'things' that the $\hat{\psi}$'s are supposed to act on will look like, i.e. states as creation operators on a vacuum, and it's a bit more work to determine commutation relations or anti-commutation relations from this set-up. Notice also that we really needed to use the Heisenberg picture perspective to achieve this interpretation.
When we go to relativistic quantum mechanics the above interpretations should not only still hold in the non-relativistic limit directly, but there is no reason why we can't consider all stationary states to be of the above form $\psi_n(x) e^{-i E_n t}$ as before, and so directly carry over the previous interpretation. The only difference here is that, due to relativity, even for free particles we now find that $E = \pm |E|$ is possible, i.e. energy seems to be either positive or negative, while in the non-relativistic case the energy of a free particle is very importantly a positive quantity. In non-relativistic quantum mechanics this happens all the time, negative energy eigenvalues just indicate a discrete spectrum (while positive energy eigenvalues indicate the continuous spectrum), but the negative energy discrete spectrum actually only occurs in a non-relativistic potential and the discrete spectrum conclusion for negative energies explicitly relies on the fact we are not studying a free particle because it's energy is positive.
Thus, for relativistic quantum mechanics, when we try to consider even the case of a free particle, we still unavoidably seem to find positive and negative energy eigenvalues, and simply can't ignore the stationary states associated to the negative energy eigenvalues. The most common approach is to always interpret the energy $E$ of a free particle as a positive quantity, and instead interpret the quantum field operator associated to the general expansion of a single-particle wave function in the stationary states
$$\hat{\psi}(\mathbf{r},t) = \sum_{\mathbf{p}} \hat{a}_{\mathbf{p}}^{(-)} \psi_{(E_{\mathbf{p}},\mathbf{p})}(\mathbf{r}) e^{-i E_{\mathbf{p}}t} + \sum_{\mathbf{p}} \hat{a}_{\mathbf{p}}^{(+)\dagger} \psi_{(-E_{\mathbf{p}},\mathbf{p})}(\mathbf{r}) e^{+i E_{\mathbf{p}}t}$$
as not only just annihilating particles (the first sum) as in the non-relativistic case, but also creating particles (the second sum). The sum really means a sum over $\mathbf{p}, s, ...$ whatever the stationary states depend on (i.e. an integral over momentum $\mathbf{p}$ and a sum over spin $\sigma$ etc...).
Also, although we are setting up a second quantized theory of a system of identical particles, there is some freedom in that $\hat{a}_{\mathbf{p}}^{(-)}$ can in general be annihilating one type of particle, and the operator $\hat{a}_{\mathbf{p}}^{(+)\dagger}$ can be (related to an operator that is) creating a different type of particle (anti-particles), but the two types of systems of identical particles should have the same mass as they came from the same Klein-Gordon/Dirac/... equation. This also allows the special case where one is in fact only deaaling with one species of particle, thus the field is called a neutral field ([2] Sec. 2, 12, 14, 21).
One can further argue on symmetry grounds that a free particle stationary state should always have, using the $(+,-,-,-)$ metric, a
$$e^{-i p x} = e^{-i(E_{\mathbf{p}} t - \mathbf{p} \cdot \mathbf{r})}$$
dependence in them, where the overall $-i$ sign is just a choice to agree with the non-relativistic case of having $e^{-iEt}$ time dependence, so that the above general expansion can be written as
$$\hat{\psi}(\mathbf{r},t) = \sum_{\mathbf{p}} \hat{a}_{\mathbf{p}}^{(-)} \psi_{(E,\mathbf{p})} e^{-i(E_{\mathbf{p}} t - \mathbf{p} \cdot \mathbf{r})} + \sum_{\mathbf{p}} \hat{a}_{\mathbf{p}}^{(+)\dagger} \psi_{(-E,\mathbf{p})} e^{-i(- E_{\mathbf{p}} t - \mathbf{p} \cdot \mathbf{r})}$$
Here the $\psi_{(E,\mathbf{p})}$ and $\psi_{(-E,\mathbf{p})}$ are just the amplitudes of the stationary states, e.g. the bispinor amplitudes $u,v$ and associated normalization factors (note one should ask why we're allowed to even use normalization factors in a continuous spectrum free particle problem where the norm of an individual eigenfunction is technically infinite, see my final comments below) in the Dirac equation case. Also, the terms in the exponentials in the second sum are not even apparently relativistic as written we should really send $\mathbf{p} \to - \mathbf{p}$
$$\hat{\psi}(\mathbf{r},t) = \sum_{\mathbf{p}} \hat{a}_{\mathbf{p}}^{(-)} \psi_{p} e^{-i(E_{\mathbf{p}} t - \mathbf{p} \cdot \mathbf{r})} + \sum_{\mathbf{p}} \hat{a}_{-\mathbf{p}}^{(+)\dagger} \psi_{-p} e^{i(E_{\mathbf{p}} t - \mathbf{p} \cdot \mathbf{r})}$$
Thus we find the second term represents the creation of particles with momentum $-\mathbf{p}$ that can be interpreted as anti-particles if they are a different type of particle, or a 'neutral field' if they are the same type off particle as referenced above. In the anti-particle case they are given the new labels $\hat{a}_{\mathbf{p}}^{(+)\dagger} = \hat{b}_{-\mathbf{p}}^{\dagger}$ and the first term is similarly re-labelled $\hat{a}_{\mathbf{p}}^{(-)} = \hat{a}_{\mathbf{p}}$ to give
$$\hat{\psi}(\mathbf{r},t) = \sum_{\mathbf{p}} \hat{a}_{\mathbf{p}} \psi_{p} e^{-ipx} + \sum_{\mathbf{p}} \hat{b}_{\mathbf{p}}^{\dagger} \psi_{-p} e^{ip x}$$
or in the neutral case something like $\hat{a}_{\mathbf{p}}^{(+)\dagger} = \hat{c}_{-\mathbf{p}}^{\dagger}$ and $\hat{a}_{\mathbf{p}}^{(-)} = \hat{c}_{\mathbf{p}}$ so that
$$\hat{\psi}(\mathbf{r},t) = \sum_{\mathbf{p}} \hat{c}_{\mathbf{p}} \psi_{p} e^{-ipx} + \sum_{\mathbf{p}} \hat{c}_{\mathbf{p}}^{\dagger} \psi_{-p} e^{ip x}$$
In the case of the Dirac equation, from the explicit form of the stationary states we can re-write the anti-particle case as
$$\hat{\psi}(x)=\int \frac{d^3p}{(2\pi)^3} \frac{1}{\sqrt{2E_\mathbf{p}}}\sum_s \left( \hat{a}_\mathbf{p}^s u^s(p)e^{-ip\cdot x} + \hat{b}_\mathbf{p}^{s\dagger}v^s(p)e^{ip\cdot x} \right); \tag{3.99}$$
Hopefully it's completely obvious now that arbitrarily deciding to interpret whatever operator is attached to say $e^{+ipx}$ (in the $(+,-,-,-)$ metric) in $\hat{\psi}$ as an annihilation operator would result in a time dependence that just completely disagrees with the discussion I started from - the usual second quantization time dependence for quantum mechanical single particle transitions that increase the number of particles in that stationary state.
You should be able to figure out the interpretation of $\hat{\psi}^{\dagger}$ based off this, or rather $\hat{\overline{\psi}} = \hat{\psi}^{\dagger} \gamma^0$. The notation/conventions with the $u$'s and $v$'s is to do with choices made in finding the stationary states of the Dirac equation in the first place, it is hopefully clear that it is the propagating modes $e^{\pm i px}$ (and the choice of metric if one uses this notation) that is vitally important in the interpretation we are being consistent.
Note how the relativistic free particle comments are inherently based in the continuous spectrum, every step of it involves physically interpreting a free particle as having the energy associated to a given stationary state. Yet it is a very common belief, even on this site, that a rigorous approach says one cannot physically interpret free particle eigenfunctions.
If taken seriously, this is saying that it's okay to throw out a physical interpretation for every single continuous spectrum free particle eigenfunction, but presumably also saying it's not okay to throw away a physical interpretation of the 'negative energy' free particle eigenfunctions in the relativistic case because of their absolutely historic importance, one should judge for themselves.
References:
- Landau and Lifshitz, "Quantum Mechanics", 3rd Ed.
- Landau and Lifshitz, "Quantum Electrodynamics", 2nd Ed.
