Why does entropy arise from order?

Question

Why does entropy arise from order?

Thanks.

I'm not sure if this question has already been answered on this site. If anyone needs me to clarify the question, please post in the comments and I'll provide an edit.

Answer 1

Good question!

First, I wouldn't quite say entropy arises from order. It's more like the tendency of systems to go from an ordered, defined, structured sort of state (like the moment when you first pour milk into coffee) to a more uniform state, equilibrium (like when the milk has fully dispersed through the coffee).

The reason for this has to do with phase space. Think about the pieces of a jigsaw puzzle. There's only one way they can be correctly placed, right? Now, there's a ton of different ways they can be incorrectly placed. That's kind of like phase space - it's the different ways a situation can play out, sort of. More specifically, it's all the possible states of system.

So, let's think about a cloud of steam. When you pass your hand through it, it doesn't really look much different, right? It is high entropy. Why? Because there are so many particles of steam, if you rearrange a couple, it looks the same. Ludwig Boltzmann knew this. He said that systems went from low entropy to high entropy because there were many more unordered states than ordered states. It's a matter of probability.

Now, because it is a matter of probability, sometimes things move from high entropy to low entropy. This is called a fluctuation. This is really unlikely, which is why we don't see things spontaneously appear, or reassemble. While we drop a mug and it breaks, we sure don't see mugs coming back together and leaping into our hands. This is the sort of thing that happens over timescales orders of magnitude larger than the length of the universe. In other words, you're not going to see anything spontaneously reassemble anytime soon.

But the probability still stands. There are so many more possibilities for systems in phase space that are high entropy that it is incredibly more likely for something to move from low entropy to high entropy. This, of course, leaves us with the question of why the universe started with low entropy, which is still a question today in cosmology.

Now, let's apply this to Boltzmann's formulation of entropy,

$$S = k \quad log \quad W$$

Where $S$ is the entropy of a system, $k$ is Boltzmann's constant, and $W$ is the number of microstates in a macrostate. This is what we were talking about earlier with phase space. Microstates is the number of ways we can rearrange, say, the molecules in a steam cloud, so that the overall steam cloud looks exactly the same (this overall picture of the steam cloud would be the macrostate). So, when there are many microstates within a given macrostate, the system is higher entropy. When there are less microstates within a given macrostate, the system is lower entropy.

To illustrate the second part, think about a couple of wooden toy blocks - maybe four. There really aren't that many ways that you can arrange them so they are different but look the same (for example, if the macrostate was having all four in a vertical stack, you could maybe move the bottom block to the top, but you can't take the top block off). So the system is low entropy.

Interestingly, this is why entropy has such a large connection to information theory - the number of ways you can make a message make sense, or contain large amounts of information, are very similar to the number of microstates in a macrostate.

Hope this helps!

Resources:

For learning about the entropy arrow of time and the cosmological aspect of entropy, I'd recommend From Eternity to Here by Sean Carroll. This website will help explain more about entropy. This website has information (pun not intended) about information theory. Finally, here is a website with information about black hole entropy, which is just really interesting. This question and this question are about the thermodynamic arrow of time.