The confusion stems from not fully understanding the difference between a Lie group element and a Lie algebra element.
The Lie group $SU(2)$ is the set of all $2\times 2$ unitary matrices $U$ with unit determinant,
$$
U^\dagger U = 1 , \qquad \det U = 1 .
$$
The Lie algebra $su(2)$ is the set of all $2\times 2$ hermitian matrices $S$ with vanishing trace.
$$
S^\dagger = S , \qquad \text{Tr}\,S = 0 .
$$
The exponential map gives the relationship between the Lie algebra and Lie group
$$
U = \exp (i S ).
$$
The phrasing used in physics is that "$S$ is a generator of the group $SU(2)$" even though $S$ is an element of the algebra $su(2)$.
The Lie algebra $su(2)$ has infinitely many finite-dimensional unitary representations (if you don't exactly know what a representation is, you can forget about this now) labeled by a half-integer $j$. The spin operators in $j=1/2$ representations are given by the Pauli matrices,
$$
S_x^{(1/2)} = \sigma_1 , \qquad S_y^{(1/2)} = \sigma_2 , \qquad S_z^{(1/2)} = \sigma_3 .
$$
Each of these matrices satisfies the criteria required for $su(2)$, i.e. they are Hermitian AND traceless. Note also that their determinants are NOT 1 and indeed, they are not supposed to be since they are elements of the Lie ALGEBRA, not the GROUP (that they happen to be unitary is pure coincidence). The corresponding GROUP element can be obtained by exponentiating these matrices. For instance,
$$
U_x = \exp ( \frac{i}{2}\theta_x S_x^{(1/2)} ) = \begin{bmatrix}
\cos \frac{\theta_x}{2} & i \sin \frac{\theta_x}{2} \\ i \sin \frac{\theta_x}{2} & \cos \frac{\theta_x}{2}
\end{bmatrix}
$$
The group element $U_x$ is unitary AND has determinant 1 as required.
The same thing extends to all the higher spin $j>1/2$ representations as well.
