I want to replace the masses with their relativistic mass where relativistic mass is the following equation:
Yes, you can. But remember that you had better write the relativistic law of momentum conservation. When you do, you will find that it is as if you replace the masses in your classical equation with the corresponding relativistic masses. However, remember that there is not only one gamma factor in your equation but rather you will have three different gammas:
$$p_1+p_2=P\space,$$
where we denote by $p_i$ the momentum of the $i^{th}$ object before the collision, and by $P$ the momentum of the stuck objects, combining as one object through an inelastic collision, after the collision. Therefore, you have:
$$\gamma_{V_1}M_1V_1+\gamma_{V_2}M_2V_2=\gamma_{V}M'V\space.$$
Also remember that, in inelastic collisions, you are not allowed to judge the final mass of the combination to be $M_1+M_2$ because additional energy changes into matter, which consequentially changes the rest mass of the combination ($M'$) to something greater than $M_1+M_2$. Therefore, you need the conservation of the relativistic total energy (before and after are the same as in the previous equation) to help you to find $M'$ via the following equation:
$$\gamma_{V_1}M_1c^2+\gamma_{V_2}M_2c^2=\gamma_{V}M'c^2\space.$$
You need both of the above equations to solve $M'$ and $V$.