Let's say you've chosen some unit of mass, the jilogram, and a unit of force, the Mewton. Let's denote by $F_M$ the force measured in Mewtons, and by $m_j$ the mass measured in jilograms. Let's say that in these units, you find
$$
F_M=4m_ja
$$
This is not Newton's law! But maybe we can make it work. For example, if we define a new unit of force, the Newton, such that (1 Newton)=(4 Mewtons), then you'll agree that if we measure force in Newtons, we'll have $F_N=F_M/4$. Thus, $F_M=4F_N$. If we substitute this in to our first equation, we find
$$
4F_N=4m_ja
$$
or
$$
F_N=m_ja
$$
So we've recovered Newton's law, by choosing a different unit of force!
Now, let's say we don't want to redefine force. Perhaps we think the Mewton is a really good unit of force, for some reason. Then we can still choose a new unit of mass. Let's define a kilogram by (4 kilograms)=(1 jilogram). Then if we measure things in kilograms, we'll have that $m_k=4m_j$. Substituting this in, we get
$$
F_M=m_ka
$$
which is again Newton's law.
Of course, if you randomly chose to measure mass in jilograms and force in Mewtons, and didn't want to redefine either force or mass (or distance or time), you'd be stuck with a $k\neq 1$. But most systems of units define three of the four quantities (length, time, force, mass), and then choose to define the last one to make Newton's law have $k=1$, just like we did above.