Measurement is not unitary, because it does not implement a unitary. It selects a unitary.
We can certainly say that a preparation $|\psi_i\rangle$ and (renormalized)
measurement outcome $|\psi_f\rangle$ are related by a unitary transformation $$|\psi_f\rangle = U_{fi} |\psi_i\rangle,$$ where $U_{fi} \in U(2)$. However, we can only say this once a measurement has occurred.
Before measurement, the possible outcomes are $|{+}\psi_f\rangle$ and $|{-}\psi_f\rangle$, so all we can say is that $|\psi_i\rangle$ and the not-yet-measured outcome $|\psi_f\rangle$ are related by either $U_{fi}^+$ or $U_{fi}^-$, where
$$|{+}\psi_f\rangle = U_{fi}^+ |\psi_i\rangle \hspace{1em} \text{and} \hspace{1em} |{-}\psi_f\rangle = U_{fi}^- |\psi_i \rangle.$$
Formally, we can construct an equivalence class of unitaries $U_{fi}^+ \sim U_{fi}^-$ over the possible outcomes of the measurement process, so that $[U_{fi}^+] = [U_{fi}^-] \in U(2)/\mathbb Z_2$ represents the part of the relation between the initial and final state that is fixed by our experimental setup.
The point is that the measurement process implements a transition
$$
U(2)/\mathbb Z_2 \to U(2) \\
[U_{fi}^\pm] \mapsto U_{fi}^\pm
$$
that breaks the $\mathbb Z_2$ symmetry and selects a particular representative of $[U_{fi}^\pm]$. In other words, the unitary relation $U_{fi}^+$ or $U_{fi}^-$ between an initial state and measurement outcome is not a description of the measurement process. It is the result of the measurement process.