If a system of particles is bound, then it has negative energy relative to the same system disassembled into its separated parts. In the nonrelativistic limit, this negative energy is small compared to the sum of the masses of the constituent particles, so the mass of the bound system is still positive.
But relativistically, there is no obvious reason why this has to be true. For example, the electromagnetic charge radius of the pion is about 0.7 fm. A particle-in-a-box calculation for two massless particles in a box of this size gives a kinetic energy of about 1500 MeV, but the observed mass of a pion is about 130 MeV, which suggests an extremely delicate near-cancellation between the positive kinetic energy and the negative potential energy. I see no obvious reason why this couldn't have gone the other way, with the mass coming out negative.
Is one of the following correct?
Some general mechanism in QFT prevents negative masses.
Nothing in QFT prevents negative masses, but something does guarantee that there is always a lower bound on the energy, so that a vacuum exists. If pions could condense from the spontaneous creation of quark-antiquark pairs, then we would just redefine the vacuum.
Nothing guarantees a lower bound on energy. The parameters of the standard model could be chosen in such a way that there would be no lower bound on energy. We don't observe that our universe is that way, so we adjust the parameters so it doesn't happen.
If 1, what is the mechanism that guarantees safety?
If 2, what is it that guarantees that we can successfully redefine the vacuum? In the pion example, pions are bosons, so it's not like we can fill up all the pion states.
If 3, is this natural? Do we need fine-tuning?
Are there no general protections, but protection mechanisms that work in some cases? E.g., in the case of a Goldstone boson, we naturally get a zero mass. Do the perturbations that then make the mass nonzero always make it positive?
Related: Is negative mass for a bound system of two particles forbidden?