.
Entropy, Coarse-Graining, and the Nature of Time
An expanded treatment of what entropy means when microstate information is ignored, compressed, hidden in correlations, or rendered inaccessible by horizons—with Shannon and von Neumann entropy included.
Entropy here means the information missing from a restricted description. It is often not a new substance added to the world. It is the logarithmic measure of how many fine-grained physical possibilities are compatible with the description available to some observer, subsystem, measuring apparatus, or effective theory.
That is the bridge between thermodynamics, Shannon information, von Neumann entropy, decoherence, horizons, and the idea that time becomes meaningful through interaction, information exchange, and local entropy production. A sharper formulation is: the arrow of time becomes meaningful when physical processes create stable, coarse-grained records while hiding microscopic reversibility in inaccessible correlations.
1. Microstate vs macrostate: the basic move
A microstate is the most detailed state description available in a theory. For a classical gas, a microstate would specify every particle’s position and momentum:
\[ x = (q_1,p_1,q_2,p_2,\ldots,q_N,p_N). \]A macrostate is a compressed description: pressure, volume, temperature, density profile, magnetization, charge distribution, pointer position, detector click, neural record, and so on.
So there is a many-to-one map:
\[ \text{microstate } x \longrightarrow \text{macrostate } M. \]Many distinct microscopic configurations produce the same macro-description. Boltzmann entropy measures the size of that compatible set:
\[ S_B(M)=k_B\ln |\Gamma_M|, \]where \(\Gamma_M\) is the region of phase space corresponding to macrostate \(M\).
2. Shannon entropy: uncertainty over classical alternatives
Shannon entropy is the cleanest mathematical expression of missing information. For a discrete random variable \(X\) with probabilities \(p_i\),
\[ H(X)=-\sum_i p_i\log_2 p_i. \]Its units are bits when the logarithm is base 2.
Interpretation:
\[ H(X)=\text{average number of bits needed to specify the actual outcome}. \]For a certain coin outcome,
\[ P(H)=1,\quad P(T)=0, \]we get \(H=0\). There is no uncertainty. But for a fair coin,
\[ P(H)=P(T)=1/2, \]we get \(H=1\) bit. One bit is needed to specify which outcome occurred.
For \(N\) equally likely alternatives,
\[ H=\log_2 N. \]So if a macrostate is compatible with \(N\) equally likely microstates, the missing information is \(\log_2 N\) bits.
Thermodynamic entropy is Shannon entropy with physical units:
\[ S = k_B \ln 2 \, H. \]Equivalently, using natural logarithms,
\[ S=-k_B\sum_i p_i\ln p_i. \]That is the Gibbs entropy.
3. Boltzmann, Gibbs, Shannon: three versions of one intuition
Boltzmann entropy
\[ S_B(M)=k_B\ln W. \]Here \(W\) is the number, or phase-space volume, of microstates compatible with a macrostate. This is entropy of a macrostate.
It asks: given that the system looks macroscopically like \(M\), how many microscopic ways could that be true?
Gibbs entropy
\[ S_G[p]=-k_B\sum_i p_i\ln p_i. \]This is entropy of a probability distribution over microstates. It asks: given my probability distribution over microstates, how much microscopic uncertainty remains?
Shannon entropy
\[ H[p]=-\sum_i p_i\log_2 p_i. \]This is the dimensionless information-theoretic version. The relation is:
\[ S_G = k_B\ln 2 \, H. \]If all \(W\) compatible microstates are equally likely,
\[ p_i=\frac{1}{W}, \]then Gibbs/Shannon reduces to Boltzmann:
\[ S_G=-k_B\sum_{i=1}^{W}\frac{1}{W}\ln\frac{1}{W} = k_B\ln W = S_B. \]4. Conditional entropy: entropy after coarse-graining
Let
\[ X=\text{microstate}, \]and
\[ M=f(X)=\text{macrostate}. \]If you know \(M\) but not \(X\), the remaining ignorance is the conditional entropy:
\[ H(X|M). \]For a particular macrostate \(M=m\), \(H(X|M=m)\) is the number of missing bits needed to specify the exact microstate after the macrostate is already known.
If all microstates compatible with \(m\) are equally likely and there are \(W_m\) of them,
\[ H(X|M=m)=\log_2 W_m, \]so
\[ S_B(m)=k_B\ln W_m = k_B\ln 2 \, H(X|M=m). \]This is the most precise version of the sentence: entropy appears when we coarse-grain because coarse-graining replaces \(X\), the exact microstate, with \(M=f(X)\), a many-to-one description. Entropy measures the information lost in that compression.
5. Correlations: entropy is often hidden in relationships
Entropy does not simply mean “lack of order.” Often it means lost access to correlations.
Shannon entropy obeys:
\[ H(X,Y)\leq H(X)+H(Y), \]with equality only when \(X\) and \(Y\) are independent. Define mutual information:
\[ I(X:Y)=H(X)+H(Y)-H(X,Y). \]This measures how much information is stored in the correlation between \(X\) and \(Y\).
Example: two perfectly correlated coins.
\[ P(HH)=1/2,\quad P(TT)=1/2. \]Each coin alone has entropy:
\[ H(X)=1,\quad H(Y)=1. \]But the pair has only:
\[ H(X,Y)=1, \]not 2. Once you know one coin, you know the other. The mutual information is:
\[ I(X:Y)=1+1-1=1. \]If you ignore the correlation, you falsely say the pair has 2 bits of uncertainty. If you keep the correlation, it has only 1 bit.
6. Fine-grained entropy may be conserved while coarse-grained entropy increases
This is the subtle core of the arrow of time.
In exact classical Hamiltonian mechanics, the fine-grained Gibbs entropy is conserved under microscopic dynamics. The usual route to entropy increase is coarse-graining: replace a detailed probability distribution with a smoothed one over macrostates.
So the microscopic picture is:
\[ \text{fine-grained information is preserved}. \]The macroscopic picture is:
\[ \text{accessible information decays}. \]The missing information has not necessarily been destroyed. It has become practically inaccessible.
This is why entropy is linked to irreversibility. Micro-dynamics may be reversible, but the macro-description is not.
7. Von Neumann entropy: the quantum version
In quantum mechanics, the state of a system is represented by a density matrix \(\rho\). The quantum analogue of Gibbs/Shannon entropy is von Neumann entropy:
\[ S_{\mathrm{vN}}(\rho) = -k_B\mathrm{Tr}(\rho\ln\rho). \]In bits, one often writes:
\[ H_{\mathrm{vN}}(\rho) = -\mathrm{Tr}(\rho\log_2\rho). \]If \(\rho\) has eigenvalues \(\lambda_i\), then
\[ S_{\mathrm{vN}}(\rho) = -k_B\sum_i\lambda_i\ln\lambda_i. \]So von Neumann entropy is the Shannon entropy of the eigenvalue spectrum of \(\rho\).
Important cases:
A pure state,
\[ \rho=|\psi\rangle\langle\psi|, \]has eigenvalues \(1,0,0,\ldots\), so
\[ S_{\mathrm{vN}}=0. \]A maximally mixed \(d\)-dimensional state,
\[ \rho=\frac{I}{d}, \]has eigenvalues \(1/d\), so
\[ S_{\mathrm{vN}}=k_B\ln d. \]In bits:
\[ H_{\mathrm{vN}}=\log_2 d. \]8. Shannon entropy of measurement outcomes is not always von Neumann entropy
This distinction is essential.
Take the pure qubit state:
\[ |+\rangle=\frac{|0\rangle+|1\rangle}{\sqrt{2}}. \]Its density matrix is:
\[ \rho=|+\rangle\langle+|. \]This is pure, so:
\[ S_{\mathrm{vN}}(\rho)=0. \]But if you measure it in the \(|0\rangle,|1\rangle\) basis, the outcomes are:
\[ P(0)=1/2,\quad P(1)=1/2. \]The Shannon entropy of that measurement is:
\[ H=1\text{ bit}. \]Why? Shannon entropy here refers to uncertainty about the outcome of a particular measurement. Von Neumann entropy refers to mixedness of the quantum state itself.
9. Entanglement entropy: entropy from losing access to part of a pure whole
Consider a Bell pair:
\[ |\Phi^+\rangle = \frac{|00\rangle+|11\rangle}{\sqrt{2}}. \]The full two-qubit system is pure:
\[ \rho_{AB}=|\Phi^+\rangle\langle\Phi^+|. \]So:
\[ S_{\mathrm{vN}}(\rho_{AB})=0. \]But if you only have access to subsystem \(A\), you trace out \(B\):
\[ \rho_A=\mathrm{Tr}_B(\rho_{AB}). \]This gives:
\[ \rho_A=\frac{I}{2}. \]So:
\[ S_{\mathrm{vN}}(\rho_A)=k_B\ln 2. \]Or, in bits:
\[ H_{\mathrm{vN}}(\rho_A)=1. \]The whole state is perfectly pure, but each part looks maximally mixed. The entropy is not in \(A\) alone. It is not in \(B\) alone. It is in the relation between \(A\) and \(B\), and it appears locally when one side is inaccessible.
10. Decoherence: entropy appears when the environment takes the phase information
Decoherence is the physical process by which quantum phase relations become inaccessible to local observers.
Suppose a system begins in a superposition:
\[ |\psi_S\rangle = \alpha|0\rangle+\beta|1\rangle. \]The environment begins in:
\[ |E_0\rangle. \]Interaction correlates the system with the environment:
\[ (\alpha|0\rangle+\beta|1\rangle)|E_0\rangle \longrightarrow \alpha|0\rangle|E_0'\rangle + \beta|1\rangle|E_1'\rangle. \]Globally, this may still be a pure state. But if you do not track the environment, you use:
\[ \rho_S=\mathrm{Tr}_E(\rho_{SE}). \]When \(|E_0'\rangle\) and \(|E_1'\rangle\) become nearly orthogonal, the reduced state of \(S\) loses its interference terms:
\[ \rho_S \approx |\alpha|^2|0\rangle\langle0| + |\beta|^2|1\rangle\langle1|. \]That looks like a classical probability distribution.
So in decoherence:
\[ \text{quantum coherence} \longrightarrow \text{system-environment correlation} \longrightarrow \text{local entropy}. \]Nothing mystical is necessarily destroyed. Information becomes delocalized into the environment.
11. Why this matters for time
A closed global quantum state evolving unitarily does not necessarily increase its von Neumann entropy. For unitary evolution,
\[ \rho(t)=U(t)\rho(0)U^\dagger(t), \]the eigenvalues of \(\rho\) are unchanged, so
\[ S_{\mathrm{vN}}(\rho(t))=S_{\mathrm{vN}}(\rho(0)). \]At the most exact quantum level:
\[ \text{unitary evolution preserves total von Neumann entropy}. \]But subsystems become entangled with other subsystems, and their reduced entropies can increase:
\[ S(\rho_A),\quad S(\rho_B),\quad S(\rho_{\text{observer}}),\quad S(\rho_{\text{environment}}). \]This gives the experienced arrow of time:
\[ \text{more records}, \quad \text{more decoherence}, \quad \text{more inaccessible correlations}, \quad \text{more coarse-grained entropy}. \]12. Entropy and the Unruh/Rindler horizon
This is where the acceleration-temperature argument becomes relevant.
In the Unruh effect, the inertial Minkowski vacuum is globally a vacuum state. But a uniformly accelerated observer has access only to a Rindler wedge of spacetime. The other wedge is causally inaccessible.
The standard result is:
\[ \rho_R = \mathrm{Tr}_L \left( |0_M\rangle\langle 0_M| \right), \]where \(L\) is the inaccessible left Rindler wedge and \(R\) is the accessible right Rindler wedge. The reduced state \(\rho_R\) is thermal.
Restoring constants, the Unruh temperature is:
\[ T_U=\frac{\hbar a}{2\pi c k_B}. \]This is not ordinary ignorance like “I forgot where I put my keys.” It is physical restricted access. The accelerated observer literally cannot access the full set of field degrees of freedom across the horizon.
So the Unruh effect is a perfect example of the phrase: entropy appears when we lose access to microstate information.
13. Entropy depends on the accessible algebra of observables
In quantum theory, a state gives expectation values for a chosen set of observables. If you restrict the observables, different density matrices may become indistinguishable.
Entropy is not purely subjective, but it is description-relative:
\[ \text{entropy relative to what observables?} \] \[ \text{entropy relative to what subsystem?} \] \[ \text{entropy relative to what horizon?} \] \[ \text{entropy relative to what coarse-graining?} \]For an omniscient global description, entropy may be low or constant. For an embedded observer with limited access, entropy can be high and increasing.
14. The relation to “time as interaction”
The attached argument says roughly: no interaction, no exchange of information, no entropy increase, therefore no local time.
The mainstream-safe version would be:
One must separate three claims:
Claim A: no interaction means no entropy production
Often true locally. If a subsystem is perfectly isolated and remains pure, then its von Neumann entropy does not increase:
\[ S_{\mathrm{vN}}(\rho_S)=0 \]can remain zero.
Claim B: no entropy production means no arrow of time
Often plausible. The thermodynamic arrow requires irreversible record formation, decoherence, heat flow, coarse-graining, or some analogous increase in accessible entropy.
Claim C: no arrow of time means no time at all
This is speculative. A closed quantum system can still evolve unitarily. A system can have changing relative phases even if its total von Neumann entropy is constant. A perfectly stationary energy eigenstate, by contrast, has no observable internal change except a global phase.
So the better distinction is:
\[ \text{time as parameter} \neq \text{time as experienced change} \neq \text{time as thermodynamic arrow} \neq \text{time as records}. \]The safest deep synthesis is:
15. Entropy is not “disorder” exactly
“Disorder” is a useful metaphor, but it breaks down.
A crystal at high temperature may look ordered spatially but have thermal entropy from phonons. A black hole looks simple from the outside but has enormous entropy. A scrambled hard drive can have high Shannon entropy but low semantic usefulness. A pure quantum superposition can produce random measurement results but have zero von Neumann entropy.
Entropy is better understood as:
Or even more compactly:
16. Where did the information go?
When entropy increases, the information usually goes into one of three places.
1. Unobserved microscopic degrees of freedom
Example: heat in air molecules. You know the temperature, not every molecular velocity.
2. Correlations
Example: two subsystems become correlated, but you examine each one separately. The local entropy rises even if the joint state remains pure.
3. Inaccessible regions
Example: horizons. A Rindler observer traces out the opposite wedge. A black-hole exterior observer cannot access interior degrees of freedom.
In all three cases, entropy means:
\[ \text{the information is not available in the variables you are using}. \]17. One compact dictionary
| Concept | Formula | Meaning |
|---|---|---|
| Shannon entropy | \(H(X)=-\sum_i p_i\log_2p_i\) | Classical uncertainty in bits |
| Gibbs entropy | \(S_G=-k_B\sum_i p_i\ln p_i\) | Physical entropy of a probability distribution over microstates |
| Boltzmann entropy | \(S_B=k_B\ln W\) | Entropy of a macrostate with \(W\) compatible microstates |
| Conditional entropy | \(H(X|M)\) | Missing micro-information after macrostate is known |
| Mutual information | \(I(X:Y)=H(X)+H(Y)-H(X,Y)\) | Information stored in correlations |
| Von Neumann entropy | \(S_{\mathrm{vN}}=-k_B\mathrm{Tr}(\rho\ln\rho)\) | Quantum mixedness / entropy of a density matrix |
| Entanglement entropy | \(S(\rho_A)\), with \(\rho_A=\mathrm{Tr}_B\rho_{AB}\) | Entropy caused by tracing out part of a pure quantum whole |
| Thermal entropy | \(S=-\partial F/\partial T\) or \(S=-k_B\mathrm{Tr}\rho\ln\rho\) for thermal \(\rho\) | Entropy tied to heat, work, and equilibrium |
18. The final philosophical compression
In this context, entropy is not merely “messiness.”
It is the quantitative shadow cast by limited access.
A god’s-eye exact description may see a pure state evolving unitarily. An embedded observer sees subsystems, macrostates, records, clicks, temperatures, horizons, and memories.
The gap between those descriptions is entropy.
So the sentence—“Entropy often appears when we coarse-grain, ignore, or lose access to microstate information”—means:
And in the time/acceleration context:
Interaction creates correlations. Correlations create records. Records require coarse-graining. Coarse-graining creates entropy. Entropy gives the arrow. The arrow gives the experienced sequence we call time.
Sources and context
This HTML export preserves the conceptual structure of the prior answer and its scientific source trail. External references are included for orientation.
- Claude Shannon, “A Mathematical Theory of Communication,” for Shannon entropy and information measured in bits: PDF.
- Stanford Encyclopedia of Philosophy, “Philosophy of Statistical Mechanics,” for Boltzmann entropy, macroregions, and coarse-graining: entry.
- Stanford Encyclopedia of Philosophy, “Information Processing and Thermodynamic Entropy,” for Gibbs entropy and coarse-graining: entry.
- Review material on quantum entropy and von Neumann entropy: arXiv PDF.
- W. H. Zurek, “Decoherence, einselection, and the quantum origins of the classical,” for decoherence, environment-induced monitoring, and record formation: PDF.
- Crispino, Higuchi, and Matsas, “The Unruh effect and its applications,” for the Rindler-wedge tracing interpretation of the Unruh effect: PDF.
- Nature Communications article on Page-Wootters-style relational time: article.
- User-provided source note: the attached text summarized McCulloch’s claim that time is emergent from change and interaction, and that low-interaction systems may have an enlarged “now.”
