While this is an old question, I imagine many encounter geometric algebra through David Hestenes's writing on zitterbewegung, so I'd like to offer a different perspective than the existing answers.
Independently of Hestenes's zitterbewegung program, his development of Dirac theory provides a clarity to what is being described in the theory that is difficult to accomplish in the usual matrix representation. This is in part due to the geometrical interpretation provided to the imaginary $i$ in the Dirac equation. While I don't know that this interpretation univocally explains the presence of complex numbers in QM, it certainly provides a way to make sense of it.
There are many algebraic objects that square negatively in a geometric algebra, and these generate rotations in distinct planes represented by those objects. So from the geometric algebra point of view, the imaginary $i$ could be thought of as a generator of rotations in a generic plane.
Let's consider a few examples in spacetime, $\mathcal{\text{Cl}}_{1,3}$.
The bivector $\gamma_2 \gamma_1$ squares to $-1$ and generates rotations in the plane spanned by the orthonormal vectors $\gamma_1$ and $\gamma_2$. This acts on vectors in the following way:
$$v \mapsto v' = e^{\gamma_2\gamma_1\phi/2} v e^{-\gamma_2\gamma_1\phi/2}.$$
Or consider the pseudoscalar $I$ which generates rotations between multivectors and their Hodge duals. In electrodynamics, the pseudoscalar generates duality rotations of the Faraday bivector $F$ via: $$F \mapsto F' = e^{I\beta/2} F e^{I \beta/2}.$$
(Yes without the minus — the reason is the general transformation law $M' = \psi M \tilde \psi$ involves reversion $\tilde \psi$ that behaves differently for these two examples: $\widetilde{\gamma_2\gamma_1} = - \gamma_2\gamma_1$ but $\tilde I = I$.)
As an aside (in response to Ron's comment), the symmetric stress-energy tensor in electrodynamics is given by
$$T(n) = \frac{1}{2} F n \tilde F,$$
which represents the flow of energy-momentum through the surface perpendicular to $n$, and describes a reflection of $n$ across the bivector $F$. I personally find this to be clearer and more manageable than:
$$T^{\mu\nu} = F^{\mu\alpha}F^{\nu}{}_{\alpha} - \frac{1}{4} \eta^{\mu\nu}F_{\alpha\beta}F^{\alpha\beta} = \gamma^\mu \cdot T(\gamma^\nu),$$
so I think it's misleading to say that this formalism "has a hard time with symmetric tensors," even if tensors form a larger algebra.
There are also bivectors that square positively and generate hyperbolic rotations (Lorentz boosts) in spacetime. For instance, take the timelike bivector $\gamma_3 \gamma_0$, which squares to $+1$. This boosts vectors in the $\gamma_3$ direction in the "$\gamma_0$-frame" as:
$$v \mapsto v' = e^{\gamma_3\gamma_0\phi/2} v e^{-\gamma_3\gamma_0\phi/2}.$$
General Lorentz transformations are of the form $R = e^{B/2}$ where $B$ is a spacetime bivector, with vectors transforming as
$$v \mapsto v' = R v \tilde R = e^{B/2} v e^{-B/2}.$$
Here's the point. The dynamical quantity of Dirac theory (a spinor) is precisely a dilation, duality, and Lorentz transformation given by
$$\psi = (\rho e^{I \beta})^{1/2} R.$$
The Dirac equation
$$\nabla \psi \gamma_2 \gamma_1 = \psi p_0$$
describes the dynamics of a spinor that rotates, dilates, and boosts the properties of an electron from its rest frame into some other frame.
In particular, $\psi$ transforms the electron's momentum in its rest frame $p_0 = m \gamma_0$ into a conserved probability current density
$$m J = \psi p_0 \tilde \psi = \rho R p_0 \tilde R$$
and its spin $S_0 = \gamma_2 \gamma_1$ (a bivector) into
$$S = \psi S_0 \tilde \psi = \rho R S_0 \tilde R e^{I \beta}.$$
The isomorphism between the above equation and the matrix representation of the Dirac equation
$$i \gamma^\mu \partial_\mu \lvert \psi \rangle = m \lvert \psi \rangle$$
is given by
$$\gamma^\mu \lvert \psi \rangle \leftrightarrow \gamma^\mu \psi \gamma_0$$
$$i \lvert \psi \rangle \leftrightarrow \psi \gamma_2 \gamma_1,$$
$$\gamma_5 \lvert \psi \rangle = -i \gamma_0\gamma_1\gamma_2\gamma_3 \lvert \psi \rangle \leftrightarrow \psi \gamma_3 \gamma_0,$$
such that the spin bivector $S$ is the observable associated with the imaginary $i$ in the matrix representation. (see p279 of Doran and Lasenby's Geometric Algebra for Physicists, a textbook that I recommend if you're interested in digging in)
This allows us to view U(1) gauge transformations as rotations associated with a particular spatial plane, rather than the complex plane, since $S_0$ is the generator of the gauge symmetry of Dirac theory in this context. Namely, observables (e.g. probability current density and spin density) are invariant under spatial rotations in the plane spanned by the vectors $\gamma_1$ and $\gamma_2$, when performed in the electron's rest frame.
This representation of U(1) as rotations in a plane is already implicit in the standard Hilbert space formalism for a single qubit. The particular plane is of course a matter of convention, equivalent to a choice of reference state relative to which all other states are defined. In particular, any state $\lvert \psi \rangle$ can be written
$$\lvert \psi \rangle = U_\psi \lvert 0 \rangle.$$
The fact that $\lvert 0 \rangle = \sigma_z \lvert 0 \rangle$ is an eigenstate of the Pauli operator $\sigma_z$ implies that
$$e^{i \phi} \lvert \psi \rangle = U_\psi e^{i \sigma_z \phi} \lvert 0 \rangle$$
which shows that U(1) phase transformations are represented in SU(2) by rotations in the spin plane defined by a given reference state — in this case, rotations in the $i \sigma_z$ plane, which in $\mathcal{Cl}_{1,3}$ is represented by $\gamma_2 \gamma_1 = I \sigma_z$.
I certainly find this to be helpful in understanding the role of complex numbers in Dirac theory, so if you find yourself wondering, geometric algebra is a good language to wonder in.
