Note: I think my answer below may be incorrect, at least as far as dark matter is concerned. It seems Kyle Oman has studied the issue further since asking this question and given an answer to a different question about dark matter collapse here, if I'm understanding correctly his answer says that for an ideal fluid with kinetic energy $K$ and gravitational energy $W$, the Jeans equations say that it only "becomes virialized" and stops collapsing when $2K$ becomes approximately equal to $-W$, which meaning it is not virialized (though it does still obey the virial theorem, see Kyle's comment below) when $2K < -W$. And John Baez's derivation did assume the ball of ideal gas is virialized, so his demonstration that collapse of an ideal gas decreases the entropy presumably wouldn't apply to a non-virialized collection of dark matter with $2K < -W$, so I presume this means it could collapse without contradicting the second law, and without the dark matter particles needing to radiate or interact as I suggested in my original answer.
If we want to examine gravitational collapse from a statistical mechanics point of view, we find that there's a tradeoff between the fact that a more spread-out collection of matter has more possible position states, whereas a more concentrated collection has more possible momentum states (because more of the system's potential energy has been converted to kinetic energy and thus the particles have higher average velocity/momentum). And in statistical mechanics, entropy is a function of the total number of states available, with higher entropy = more possible states. It turns out, though, that this tradeoff alone is not enough to explain why gravitational collapse can happen in some systems--the decrease in the number of possible position states when a cloud collapses is actually greater than the increase in the number of momentum states, as derived on this page from physicist John Baez, so if these were the only factors at play the entropy would be lower in the collapsed state than the diffuse state, and gravitational collapses would never occur. However, it turns out that if the collapsing matter can radiate energy away as it collapses, in that case the end state of "more concentrated, hotter matter distribution + outgoing radiation" can have a higher entropy than the initial state of "more spread out matter which hasn't yet radiated", and so this is the key to understanding why gravitational collapse respects the 2nd law of thermodynamics. As explained by Lubos Motl in this answer:
If you didn't allow the molecules to emit photons when they collide,
they wouldn't ever shrink spontaneously by obeying the laws of
gravity. The probability that a molecule slows down (or gets closer)
under the gravitational influence of the other molecules would be
equal to the probability that it speeds up (or gets further) - in
average. If you introduce some objects and terms in the Hamiltonian
that allow inelastic collisions, these inelastic collisions will
selectively slow down the molecules that happened to be closer to each
other, which is the mechanism that will be reducing the average
distance between the molecules (the actual rate will depend on the
gravitational attraction, too).
I wrote photons because, obviously, the probability of the emission of
a photon is much higher for real-world gases because most of their
interactions are electromagnetic interactions. Because a photon
carries as much entropy as a graviton would, but you produce many more
photons by random collisions, the entropy increase is stored in the
photons. The entropy carried by gravitons is smaller by dozens of
orders of magnitude.
And as explained in this answer by Ted Bunn, this is relevant to why dark matter would "clump" only very weakly (as seen in detailed physical simulations like the ones I have linked to)--dark matter particles would only experience irreversible interactions with other particles very rarely, from either occasional interactions involving the weak nuclear force (which would be infrequent, as with neutrinos which normally pass straight through the Earth, with only about 1 in 10^11 interacting with any of the particles that make up the Earth according to this page) or shedding gravitons:
But it's true that dark matter doesn't seem to have collapsed into
very dense structures -- that is, things like stars and planets. Dark
matter does cluster, collapsing gravitationally into clumps, but those
clumps are much larger and more diffuse than the clumps of ordinary
matter we're so familiar with. Why not?
The answer seems to be that dark matter has few ways to dissipate
energy. Imagine that you have a diffuse cloud of stuff that starts to
collapse under its own weight. If there's no way for it to dissipate
its energy, it can't form a stable, dense structure. All the particles
will fall in towards the center, but then they'll have so much kinetic
energy that they'll pop right back out again. In order to collapse to
a dense structure, things need the ability to "cool."
Ordinary atomic matter has various ways of dissipating energy and
cooling, such as emitting radiation, which allow it to collapse and
not rebound. As far as we can tell, dark matter is weakly interacting:
it doesn't emit or absorb radiation, and collisions between dark
matter particles are rare. Since it's hard for it to cool, it doesn't
form these structures.
Detailed cosmological simulations like the "Illustris simulation" discussed in this article and this one indicate that there is some clustering with dark matter, but it doesn't form very condensed clumps on the scale of stars.
