When doing scalar QFT one typically imposes the famous 'canonical commutation relations' on the field and canonical momentum: $$[\phi(\vec x),\pi(\vec y)]=i\delta^3 (\vec x-\vec y)$$ at equal times ($x^0=y^0$). It is easy (though tedious) to check that this implies a commutation relation for the creation/annihilation operators $$[a(\vec k),a^\dagger(\vec k')]=(2\pi)^32\omega\delta^3(\vec k-\vec k') $$
When considering the Dirac (spinor) field, it is usual (see e.g. page 107 of Tong's notes or Peskin & Schroeder's book) to proceed analogously (replacing commutators with anticommutators, of course). We postulate \begin{equation} \{\Psi(\vec x),\Psi^\dagger(\vec y)\}=i\delta^3(\vec x -\vec y) \end{equation} and, from them, derive the usual relations for the creation/annihilation operators.
I'd always accepted this and believed the calculations presented in the above-mentioned sources, but I suddenly find myself in doubt: Do these relations even make any sense for the Dirac field? Since $\Psi$ is a 4-component spinor, I don't really see how one can possibly make sense out of the above equation: Isn't $\Psi\Psi^\dagger$ a $4\times 4$ matrix, while $\Psi^\dagger\Psi$ is a number?! Do we have to to the computation (spinor-)component by component? If this is the case, then I think I see some difficulties (in the usual computations one needs an identity which depends on the 4-spinors actually being 4-spinors). Are these avoided somehow? A detailed explanation would be much appreciated.
As a follow-up, consider the following: One usually encounters terms like this in the calculation: $$u^\dagger \dots a a^\dagger\dots u- u\dots a^\dagger a \dots u^\dagger $$ Even if one accepts that an equation like $\{\Psi,\Psi^\dagger\}$ makes sense, most sources simply 'pull the $u,\ u^\dagger$ out of the commutators' to get (anti)commutators of only the creation/annihilation operators. How is this justified?
EDIT: I have just realized that the correct commutation relation perhaps substitutes $\Psi^\dagger$ with $\bar \Psi$ (this may circumvent any issue that arises in a componentwise calculation). Please feel free to use either in an answer.