There are various threads on this site explaining the mathematical details of how, in QFT, position operators are non-relativistic. I can follow some of the math, while some of it goes over my head. But even with the parts of the math I can follow, I have a hard time relating it to anything conceptual -- that is, all the mathematical explanations I've seen have a feel similar to proofs by contradiction in that they show that it's true there are no relativistic position operators, but not why it's true. So, to be clear, what I'm looking for isn't a nonmathematical explanation necessarily, but just an explanation that feels like more than just a bunch of algebraic manipulations -- something that connects all those calculations to the actual objects we're trying to model and their properties.
To clarify what specifically I'm confused about, I don't get why bringing in special relativity should mess up the notion of position at all. I understand that position is a relative property, so that if we change reference frames any position operator would be affected. But momentum and energy are also relative, and yet there doesn't seem to be any issue with defining relativistic versions of their operators. What makes position so different that we can't just define it in some given reference frame and apply the Lorentz transform as needed? Does it have something to do with the geometry of spacetime being hyperbolic?