If we write something like:
$\partial_a X_{\mu} \partial^a X^{\mu}$
Does that mean the first derivative is only applied to the first X?
($\partial_a X_{\mu})( \partial^a X^{\mu}$)
Or is the first derivative applied to the object $X_{\mu} \partial^a X^{\mu}$, such that second derivatives would appear?
It is definitely an ambiguous notation, but one that is quite conventional. You should interpret it as: $(\partial_a X_\mu)(\partial^a X^\mu)$. For instance, often the Klein-Gordon Lagrangian is written as: $$\mathcal{L} = \frac{1}{2} \partial_\mu \phi \partial^\mu \phi + \cdots $$ which should be interpreted as: $$\mathcal{L} = \frac{1}{2} (\partial_\mu \phi)( \partial^\mu \phi)+ \cdots $$
In my experience, the correct interpretation is $$ \partial_\mu X \partial^\mu X = (\partial_\mu X)( \partial^\mu X). $$
Total derivatives are usually written clearly as $$ \partial_\mu(\ldots). $$
