It would be more correct to state Newton's second law as following:
$$\vec F = \frac{d \vec p}{dt}$$
This holds in relativistic mechanics too:
$$\vec F = \frac{d \vec p}{dt} = \frac{d (\gamma m \vec v)}{dt} = \gamma^3 m \vec a_{//}+\gamma m \vec a_{\perp}$$
Where $\vec a_{//}$ is the component of the acceleration which is parallel to $\vec v$ and $\vec a_{\perp}$ is the orthogonal component. So the form $\vec F = m \vec a $ doesn't hold in relativistic mechanics.
Moreover, as AccidentalFourierTransform pointed out, relativistic mass is an obsolete concept. The $m$ I used in the above formula is a constant. It is $\gamma$ that is a function of velocity:
$$\gamma = \frac{1}{\sqrt{1-(v/c)^2}}$$
In old times people used to define the "relativistic mass" as
$$ m_r =\gamma m = \frac{m}{\sqrt{1-(v/c)^2}}$$
But even if we wanted to use this concept, $\vec F = m_r \vec a$ would not hold. You can verify that starting from
$$\vec F = \frac{d \vec p}{dt} = \frac{d (\gamma m \vec v)}{dt}$$
Since $\gamma$ is a function of $\vec v$ and $\vec v$ is a function of time, you cannot take $\gamma$ and $m$ out of the time derivative.
So the answer is that $\vec F = m \vec a$ just doesn't hold in relativistic mechanics.