17

What is the relationship between Energy, Entropy, and Information?

I read this - What Is Energy? Where did it come from? - and the top answer says that 'energy' is an abstract number that is a property of nature that just so happens to be conserved because 'operations' on nature are symmetric in time (time-translational symmetry).

The other conservation law (I've heard) for an abstract number that is a property of nature is Information - as in 'Black holes must behave this way because otherwise they would violate conservation of information'. But I don't know why it's conserved.

The third abstract number that is a property of nature, but that does NOT obey conservation along the direction of time - 'Entropy'. Why isn't this conserved if the other two are?

How are these three related? Again I can't follow complex math, but these must be fundamentally related to each right?

Community
  • 1
PrasannaK
  • 171
  • Just as a comment, I don't know where you get your quote on black holes from but the idea, as I understand it, is precisely that when a black hole absorbs, say, a planet, all the features of the latter become unreachable by any mean (except its mass and possibly charge as well) and there is a huge loss of information then. This lead a student of Hawkings' to suggest that, as a consequence, the entropy of the black hole has to increase. Is that what you meant? – gatsu Oct 10 '14 at 21:18

3 Answers3

3

Entropy is a measure of the order/disorder during the transformation of the state of a system and is defined as the total variation of energy at a defined temperature. From point of view of statistical mechanics, this variation of energy is generated from statistical transitions of the internal states of the system. In this sense, entropy can measure how easy it is to reach a defined state of the system. Now, imagine a text stream that arrives to you character by character in a screen. If the text is meaningless, then every character has the same probability of appearing to you and therefore the entropy is maximal because this disorder is maximal. If you want to transfer information, then you have to spend a little bit of energy in ordering the characters because this does not happen spontaneously. The final state of the system is more ordered in respect to the earlier one, so the entropy is less than the entropy of random text. This means that, if you want to reduce entropy in order to transfer information, then you must spend energy.

Void
  • 19,926
  • 1
  • 34
  • 81
Emanuele Luzio
  • 836
  • that doesn't explain why entropy must increase with time – James Kujareevanich Jul 05 '12 at 11:19
  • Entropy measure the total variation of energy. If the system is going on a more foundamental state loose a part of energy and entropy increase, on contrary entropy is diminished. – Emanuele Luzio Jul 05 '12 at 13:02
  • 1
    Entropy is not the total variation of energy at a defined temperature. The change in entropy is the heat flow divided by the temperature. If you squeeze a gas slowly, you increase the energy without increasing the entropy. – Ron Maimon Jul 05 '12 at 15:46
1

I am going to address the question as to why energy and information have time symmetric conservation properties whereas entropy does not.

According to the Wikipedia entry on entropy - "The entropy of an isolated system never decreases, because isolated systems spontaneously evolve towards thermodynamic equilibrium, which is the state of maximum entropy."

Therefore, entropy will only increase with time in an isolated system if it has not reached thermodynamic equilibrium. The universe is an isolated system that had a very low entropy state in the past, i.e. at or shortly after the big bang. Therefore it is in the process of approaching thermodynamic equilibrium. Therefore it is a circumstance of the state of our universe that causes entropy to increase with time, at this time. It is not a law.

Conservation of energy and information are laws. Increasing entropy is a circumstance. That is why they are different.

I hope this helps. It’s hard to find straight answers to this question, which suggests that we really don’t know the answer yet.

Community
  • 1
Joe Mc Swiney
  • 11
-3

Entropy increases precisely because energy is conserved. To hold a system of particles in an ordered state requires energy. This is quantified in the potential energy of the system. That energy must have come from somewhere (because energy is conserved), and whatever other system of particles provided that energy will experience an increase in entropy corresponding (under ideal circumstances) to the amount of energy it has lost. This is why entropy does not decrease. In practice, the transfer of energy is not perfect, and some energy is lost. That is why entropy is observed to increase.

Leland Reardon
  • 1