How does entropy relate to energy, conceptually?

Question

Conceptually, I've always understood entropy to be a statistical idea. For example, if you have a vacuum inside of a box and you place a handful of gas atoms on one side, the molecules have a higher statistical probability to spread out, versus remain in one concentrated spot. Therefore, on average, they will spread around and entropy is increased.

Of course there's a more elaborate definition involving macrostates and microstates, where entropy is greater when a macrostate has more microstates. However, in my understanding, the idea is still the same: increasing entropy is really just a statistical likely-hood that will hold true on average.

There are many formulas that relate entropy to energy. For example, Gibbs Free Energy: $$ΔG = ΔH - TΔS$$

Now, before I explain my confusion, I'd like to share an example from my course: hydrogen bonding (non-covalent interactions... like electrostatic interaction).

We call the formation of hydrogen bonds as being "stabilizing" - that is, they lower the overall energy $ΔG$. However, when hydrogen bonds form, the $ΔS$ actually decreases; there is more order when molecules align to form H-bonds. But this entropy cost is offset by a greater decrease in enthalpy, bringing the overall $ΔG$ down (so it's negative), and therefore making the bond "stabilizing".

That's where my confusion begins. I'm having a difficult time understanding how we can relate the idea of entropy to energy. A paper that my professor once showed me makes it clear that entropy isn't a measure of energy density/distribution... rather, it's more of a statistical observation.

So, assuming that entropy is a statistical idea (which is where I think my gap in understanding may be), how does forming a hydrogen bond, or being ordered in general, "destabilize"/increase the energy? Why does being ordered - which is simply just a small probability event actually occurring - impact the energy of a system? It's not like some outside force is inputting energy into the system to align and order the molecules... so how is entropy, a statistical idea, playing a role in the energy?

Answer 1

Here is a mathematically precise real-world example: Consider a monoatomic ideal gas of $N$ atoms in a box of constant volume $V$, in contact with an environment of temperature $T$.

Now imagine increasing the temperature by a small amount $dT$, which will increase the average total energy $U$ of the gas by an amount $dU = \frac{3}{2} N k_B dT$. The first law $dU = T\,dS - p\,dV = T\,dS$ says that the entropy will increase by $dS = \frac{dU}{T} = \frac{3}{2} N k_B \frac{dT}{T}$.

Now, as you know, entropy is $S = -k_B \sum_i p_i \ln p_i$ where the sum goes over all possible microstates $i$ of the gas and $p_i$ is the probability of microstate $i$. It is a measure of the unpredictability of the current microstate: How difficult is it to guess the microstate if you only know the macro variables?

How to interpret the entropy increase? The first thing to notice is there are more microstates with high energy than with low energy. For example, there are fewest microstates with energy $0$: All particles are standing still with $0$ energy. If you increase the energy, there are more ways to spread the energy across the $N$ particles, so it is more difficult to guess the actual microstate. If the average energy $U = \sum_i p_i E_i$ increases, the microstates with higher energy will be more likely, and since there are a greater number of them, the entropy should increase.

Apologies if that was a bit rambling...

---

Edit: Regarding your example of hydrogen bonding and Gibbs free energy. The big picture is that you have to consider both the system (the two hydrogen atoms) and the environment (the rest of the universe, including all the other molecules of the gas).

The basic problem of thermal equilibrium is like this: We know that the total energy in the universe is constant: $E_\mathrm{sys} + E_\mathrm{env} = E_\mathrm{tot}$. Furthermore, both the system and the environment have greater entropy the more energy they have. Now, the Second Law says that all processes will tend to maximise the total entropy of the universe: $$ \text{In equilibrium, } S_\mathrm{tot} = S_\mathrm{sys}(E_\mathrm{sys}) + S_\mathrm{env}(E_\mathrm{tot} - E_\mathrm{sys}) \text{ is maximised}. $$

What will $E_\mathrm{sys}$ be in equilibrium? If the system gets less energy, the entropy of the system decreases. But the environment gets more energy and can therefore have more entropy. That is why the system tends to minimise energy: because it increases the overall entropy!

But there is of course a tradeoff: If the system gives away energy, it will also lose some entropy. Equilibrium occurs when the system's lost entropy is equal to the environment's gained entropy. So from the system's point of view, there is a tension between minimising energy and maximising entropy. Minimising the Helmholtz and Gibbs free energies is a clever way to find the solution to this problem.

As an aside, we can also express the sentence in italics as $$ \frac{dS_\mathrm{sys}}{dE_\mathrm{sys}}(E_\mathrm{sys}) = \frac{dS_\mathrm{env}}{dE_\mathrm{env}}(E_\mathrm{tot} - E_\mathrm{sys}). $$ In equilibrium, the system and the environment should have the same value of $\frac{dS}{dE}$. Actually this is a definition of temperature: $\frac{dS}{dE} = \frac{1}{T}$. So it just says that at equilibrium, $T_\mathrm{sys} = T_\mathrm{env}$.

Answer 2

Say you have a box. And you have $n$ balls numbered from $1$ to $n$. Your goal is to put the balls inside the box. However, more the number of balls inside the box, the harder it is to add more. That’s just how the box is. With one box there is not much choice and we have to brave through and put all $n$ balls inside.

However, if now there are two such boxes it is easier to put $n/2$ in each box than to put all $n$ in each (or any other configuration for that matter). And similarly if you have more boxes. The least energy is expended when we share the balls equally in all boxes.

Notice however, that if now someone closes all the boxes and asks you where ball number $k$ is, you can’t say where it is. But if you had put all of them in one box then you can tell exactly where it is (by lifting the box say). In this sense, you have reduced the energy required at the expense of losing information about where the ball is. In order to minimise energy, you are spreading your system out.

This is a crude analogy for how energy is impacted by entropy. The system will always tend towards a lowest energy configuration and this happens to be the configuration where the system is spread out.

Hope this helps.