I like to answer these kinds of questions with a simple working example. So let's write a simple scalar field theory that breaks rotation but not translation invariance:
\begin{equation}
\mathcal{L}= -\frac{1}{2} \left( \eta^{\mu\nu} + \lambda e^\mu e^\nu \right) \partial_\mu \phi \partial_\nu \phi
\end{equation}
where $\lambda$ is a constant and $e^\mu$ is some fixed unit norm ($\eta_{\mu\nu} e^\mu e^\nu=1$) spacelike 4-vector that is constant over spacetime.
Now let's do a variation with a spacetime dependent shift $\epsilon(x)$; we will then use the trick that because translations with $\epsilon={\rm const}$ are a symmetry we can always write the variation as $\delta \mathcal{L}=-\partial_\mu \epsilon_\nu \tilde{T}^{\mu\nu}$, which after an integration by parts is $\epsilon_\mu \partial_\nu \delta T^{\mu\nu}$ (where $\tilde{T}$ and $T$ potentially differ by a term whose divergence is identically zero). Then since $\partial_\mu T^{\mu\nu}=0$ on shell, we can identify $T^{\mu\nu}$ as the stress-energy tensor.
More concretely, we write $x^\mu \rightarrow x^\mu - \epsilon^\mu(x)$, so $\phi \rightarrow \phi + \epsilon^\mu \partial_\mu \phi$ and $\partial_\mu \phi \rightarrow \partial_\mu \phi + \epsilon^\nu \partial_\mu \partial_\nu \phi + \partial_\nu \phi \partial_\mu \epsilon^\nu$, leading to
\begin{eqnarray}
%
\delta \mathcal{L}
&=&
- \frac{1}{2} \left[ \eta^{\rho\sigma} + \lambda e^\rho e^\sigma \right] \left[ \delta(\partial_\rho \phi) \partial_\sigma \phi + \partial_\rho \phi \delta (\partial_\sigma \phi) \right] \\
%
&=&
- \frac{1}{2} \left[ \eta^{\rho\sigma} + \lambda e^\rho e^\sigma \right]
\left[ \epsilon^\nu \partial_\rho \partial_\nu \phi \partial_\sigma \phi
+ \epsilon^\nu \partial_\sigma \partial_\nu \phi \partial_\rho \phi
+ \partial_\rho \phi \partial_\nu \phi \partial_\sigma \epsilon^\nu
+ \partial_\sigma \phi \partial_\nu \phi \partial_\rho \epsilon^\nu \right] \\
%
&=&
- \frac{1}{2} \left[ \eta^{\rho\sigma} + \lambda e^\rho e^\sigma \right]
\left[ \epsilon^\nu \partial_\nu \left( \partial_\rho \phi \partial_\sigma \phi \right)
- \epsilon_\mu \partial_\sigma \left(\partial^\mu \phi \partial_\rho \phi \right)
- \epsilon_\mu \partial_\rho \left(\partial^\mu \phi \partial_\sigma \phi \right) \right] \\
%
&=&
\epsilon_\mu \partial_\nu \left[ \partial^\mu \phi \partial^\nu \phi - \frac{1}{2} \eta^{\mu\nu} (\partial \phi)^2 + \lambda \left( e^\mu \partial^\nu \phi e^\rho \partial_\rho \phi - \frac{1}{2} e^\mu e^\nu (\partial \phi)^2\right) \right]\\
%
&=& \epsilon_\mu \partial_\nu \left[T_{\rm K.G.}^{\mu\nu} + \lambda T_{\rm S.B.}^{\mu\nu} \right]
\end{eqnarray}
where $T_{\rm K.G.}^{\mu\nu} = \partial^\mu \phi \partial^\nu \phi - \frac{1}{2} \eta^{\mu\nu}(\partial \phi)^2$ is the stress tensor for a massless Klein-Gordon field, and the term $T^{\mu\nu}_{\rm S.B.}$ is the stress-energy tensor due to the breaking of rotational symmetry (S.B.=symmetry breaking).
Indeed, the expression for $T^{\mu\nu}_{\rm S.B.}$
\begin{equation}
T^{\mu\nu}_{\rm S.B.} = e^\mu \partial^\nu \phi e^\rho \partial_\rho \phi - \frac{1}{2} e^\mu e^\nu (\partial \phi)^2
\end{equation}
has a non-vanishing anti-symmetric part $\sim (e \cdot \partial)\phi e^{[\mu}\partial^{\nu]} \phi $. One way to explain in words what is happening is that $\epsilon_\mu$ is never directly contracted with $e^\rho$ or $e^\sigma$, and unlike the metric tensor there is no way to "generate more" $e^\rho$ or $e^\sigma$ by raising and lowering indices. Another way is to explain the difference between the rotationally invariant and symmetry breaking terms is to point out that if we replace $e^\mu e^\rho$ with $\eta^{\mu\rho}$ in the first term of $T^{\mu\nu}_{\rm S.B.}$, the first term becomes $\partial^\mu\phi \partial^\nu \phi$, which is symmetric; the "symmetry breaking" term $\sim e^\mu e^\rho$ picks out a special direction on which the final answer can depend, which allows for an anti-symmetric part. To be honest, this isn't the result I was expecting when I started working this out (although I don't know why since there is no way angular momentum can be conserved and I agree with the logic in your question), but as far as I can tell it is correct.
It's always possible to define a symmetric stress-energy tensor by defining it as a variation of the action with respect to the metric, but I take it from the question that this is not what you are interested in.
Another question is whether it is possible to add piece that is identically divergence free to $T^{\mu\nu}_{\rm S.B.}$ to cancel the anti-symmetric part... I suspect (but am not 100% sure) the answer is yes, since I would think you should be able to derive the Einstein-Hilbert version of the stress-energy tensor in this way. (This isn't always possible but I suspect it probably is in this example)
