Entropy and information - Heat generation in computers

Question

I understand that part of the heat release from a computer comes from resistive heating, but to my understanding it is also linked to entropy changes and information - such that even a superconducting computer must generate some heat. Now I'm neither an expert on information theory, computers, or entropy, so please do correct my inevitable mishaps.

Say we have two registers, A and B, each holding one bit of information. We pass these through an OR gate, it returns only one bit of information, 0 or 1. It seems then that, in passing two bits into the gate, and getting out only one we have "deleted" one bit of information, corresponding to an entropy decrease $k_b\log(2)$.*

This means that the surroundings (the room the computer is in) must have a commensurate increase in entropy, in accordance with the 2nd Law of Thermodynamics, right? This is why the room must heat up? I'm still confused as to where exactly that energy comes from... Since energy can't be created, for the room to heat up, something else must've lost the energy... Would this simply be the electrons in the wiring/transistors?

Resistive heating I can explain to almost anyone, by analogy of electrons jostling through wires. But as for why going from 2 bits of information to 1 should heat up the room, I struggle to put it simply.

*I suppose a different way of phrasing it is that the macrostate of 2 bits (A and B) has 4 microstates, such that $S=k_b\log(4)$ but passing the OR gate leaves only 1 bit, with 2 microstates. Thus $\Delta S = k_b \left(\log(4) - \log(2)\right) = k_b \log(2)$.

Answer 1

You are right that deleting information increases entropy, according to the second law of thermodynamics. However, this does not mean that the room must heat up as a result. The heat transfer from the computer to the surroundings is not caused by the information deletion, but by the physical process of erasing the bit.

Let me explain with an analogy. Suppose you have a book with 100 pages, each containing one letter of the alphabet. You want to write a message using only one letter per page, such as "HELLO". To do this, you need to erase all the other letters on each page, leaving only one letter visible. This process takes some time and energy, which is dissipated as heat.

Now suppose you have a room with a fan and a thermometer. The fan blows air over the pages of your book, while the thermometer measures the temperature of the air. The fan and the thermometer are connected by wires that carry electricity from your computer to them.

When you erase a bit from your book, you are changing its entropy from kblog(2) to kblog(1). This means that your book has become more disordered and less predictable. However, this does not mean that your room must also become more disordered and less predictable. The room only becomes warmer because of two reasons:

• First, some of the heat generated by erasing a bit is transferred from your book to your room through conduction or convection.
• Second, some of the heat generated by erasing a bit is transferred from your room to your computer through conduction or convection.

The first reason is related to entropy increase in your book, while the second reason is related to entropy decrease in your computer.

The second reason is important because it shows that information cannot be destroyed or created in a closed system. Your computer can only store or retrieve bits of information using its memory cells. When you erase a bit from your book using an OR gate, you are not destroying or creating any information in either system. You are simply changing its state from 0 or 1 to 0 or 1 again.

Therefore, there is no contradiction between entropy increase and conservation of information in quantum mechanics.However, there may be some situations where information can be destroyed or created in an open system, such as Maxwell's demon. In these cases, there may be some violation of thermodynamics.

Answer 2

It seems then that, in passing two bits into the gate, and getting out only one we have "deleted" one bit of information, corresponding to an entropy decrease $k_b\log(2)$.*

Yes, but note this is change of information entropy of our model of the device(change of the Shannon entropy of the probability distribution for bit values), not necessarily change of thermodynamic entropy.

This means that the surroundings (the room the computer is in) must have a commensurate increase in entropy

If we believe that information cannot be destroyed, i.e., all details of the past physical state before the loss operation "OR" somehow survive also in the future physical state after the loss operation, then yes, information entropy associated with the environment increases (not necessarily the thermodynamic entropy). This is motivated by the Liouville theorem from classical mechanics and similar theorem in quantum mechanics, which states conservation of the information entropy functional during Hamiltonian evolution. This works with nice enough Hamiltonians; only special/pathological Hamiltonians (potential functions) imply evolution in time which makes the system forget some details about the past state.

This "perfect memory" of mathematical physical models certainly is consistent with 2nd law of thermodynamics, but the latter does not imply such memory. Standard 2nd law of thermodynamics even does not hold for small enough microscopic systems, but this does not mean we expect microscopic systems to violate the perfect memory condition.

So the idea that increase of information entropy also means increase of thermodynamic entropy is hard to justify, certainly 2nd law of thermodynamics does not directly imply it.