When one computes Hall conductivity $\sigma_{xy}$, one can show that the zero temperature Kubo formula gives
\begin{align} \sigma_{xy}(\omega) = -\frac{i}{\omega} \sum_{n\neq 0} \left[\frac{\langle 0|J_y |n\rangle \langle n | J_x|0 \rangle}{\hbar \omega + E_n - E_0} - \frac{\langle 0|J_x |n\rangle \langle n | J_y|0 \rangle}{\hbar \omega + E_0 - E_n}\right] \end{align}
Now, one is in many instances interested in the $\omega\rightarrow 0$ limit, in which case one Taylor expands the denominators:
\begin{align} \frac{1}{\hbar \omega + E_n - E_0} = \frac{1}{E_n - E_0} - \frac{\hbar\omega}{(E_n - E_0)^2} + \mathcal{O}(\omega^2) \end{align}
In particular, in many places (e.g. David Tong's Quantum Hall notes) they argue away the $\omega^0$ term so that there is no zero-frequency divergence. I want to know how this is explicitly done.
Another stackexchange post argued for the use of the trick $J_x \sim v_x = \frac{\partial}{\partial t} x = [H,x]$, which cancels the energy factors in the denominator and result in the desired vanishing of leading term because the commutator $[x,v_y]=0$. The problem, however, is that performing such a trick for the second term on both $J_x$ and $J_y$ gives a term proportional to the commutator $[x,y]=0$. Thus, such an argument also shows that the second order term also vanishes, which is patently false.
To argue that the leading term vanishes, Tong's lecture notes invoke parity and gauge-invariance (without explicit argument). Parity makes sense, but the fact that $\sigma_{xy} = -\sigma_{yx}$ should be a result, not a constraint, so I feel there should be a more direct way of showing that it vanishes. The statement about gauge-invariance is mysterious to me.
Why is the Heisenberg equation of motion approach to this wrong, and what is the proper way to explicitly show that the leading term vanishes?