The force of friction that you have to overcome is proportional to the "normal force" - for a horizontal surface, that is the weight of the object. It is normally given as
$$F_f = \mu F_n$$
where $\mu$ is the coefficient of static friction.
"Keep it sliding at a constant rate" implies that the acceleration is zero, so the net force is zero - that is, the force of friction equals the force used to push the object. So yes, a heavier object requires more force to "keep moving" at constant velocity. In your expression
$$\frac{F - F_f}{m}=a$$
you want $a=0$, so $F=F_f=\mu F_n$. We normally consider $\mu$ to be constant (independent of weight), at which point you get the exact result you were expecting intuitively. In reality, once the object is moving we call $\mu$ the coefficient of dynamic friction, which is usually lower than the coefficient of static friction (needed to initiate movement) and which may well be a function of velocity. See for example this earlier answer on models for contact friction. Tip of the hat to @K7PEH for the nudge to add a bit more detail about the difference between static and dynamic friction.