In the chiral model, the quark field $q=\begin{pmatrix} u \\ d \end {pmatrix}$ transforms like $q\rightarrow\exp({i(\theta_a\tau^a+\gamma_5\beta_a\tau^a)})\;q$. Now, I understand that the transformations defined by $$ U = {i(\theta_a\tau^a+\gamma_5\beta_a\tau^a)} $$ are the generators of the $SU(2)\times SU(2)$ chiral group. Now, to couple the electromagnetic fields for instance, one introduces gauge fields $A_\mu$ that transform like $$ A_\mu \rightarrow U A_\mu U^\dagger - \frac{i}{g}(\partial_\mu U)U^\dagger $$ which allows the definition of a covariant derivative so that the derivatives transform like the fields (this is the usual method also used to couple scalar fields to QED).
These transformations imposed on $A_\mu$ only work to produce the desired outcome if $U^\dagger U = 1$.
However, as explained in Schwartz's Quantum Field Theory, this property can not be true for generators of the Lorentz group, since there are not finite-dimensional spinor representations of the Lorentz group that are unitary.
How does one reconcile these two facts? Is there something I am missing?
