Dr Pournelle asked an interesting question the other day. (His blog uses the technology of 1995, making permalinks difficult, hence the link to his front page.) The question is this: Entropy always increases. Current cosmological theory has the early universe going from an undifferentiated mass of hydrogen with a bit of helium, to stars and galaxies. What accounts for this apparent decrease in entropy?

The answer is that it’s not a decrease at all, but it does require a bit of digging to see why, hence Dr Pournelle’s confusion. (His post makes it clear that he knows this, he just wants to see the problem worked out.) As usual, the definition of entropy as ‘disorder’ is at the root of the problem. It is by no means unreasonable to argue that stars and galaxies are more orderly than a universe-wide gas. Nonetheless, their entropy is in fact larger, but it is not trivial to show it. Begin by recalling the definition of entropy:

S = k_{B} ln Ω

where k_{B} is Boltzmann’s constant and not important to the argument, and Ω is the number of distinguishable microstates that the system could be in without changing its macrostate. A microstate is a description in terms of positions and momenta of particles; a macrostate is a description in terms of temperature, volume, or other thermodynamic quantities.

First, I will restate the question in its simplest terms: Given a number of particles, is the entropy higher when they are evenly distributed, or when they are clustered? (That is, clouds of gas, or stars.) As a visualisation, consider the universe as a simple two-dimensional grid, with evenly sized squares; and let us distribute particles into the squares, so that the universe can be specified as “1 particle in square 1, no particles in square 2”, and so on. (The argument is precisely the same for three dimensions and boxes, but two dimensions are easier to visualise.) To have some concrete numbers, let us say there are 100 by 100 squares, total 10000, and 10 particles.

(As an aside, the British staff faced a similar problem in WWII. V-2 weapons were hitting London daily; the question was, were the Germans able to *aim* the rockets, which would make it a good idea to evacuate sensitive targets like the Houses of Parliament; or were they hitting at random, in which case Parliament could take its chances like everyone else? Some of the planners felt that the existence of clusters (that is, multiple hits in single grid locations, with the grid superimposed on a map of London) showed aiming, but a mathematical analysis like the one I’m about to undertake demonstrated that there were no more clusters than you would expect by chance.)

The first problem is that ‘clustered’ and ‘uniformly distributed’ are not so easily defined, as will become clear. Let’s say that anything with five particles or more in one square is a clustered state. This is not the same as saying that anything which fails to meet this condition is uniform – there are grey areas. However, having half or more of the universe’s mass in one location is most certainly a cluster. How many such states exist? First, choose the square with the cluster; ten thousand options. Next, distribute the remaining n=5, 4, 3, 2, 1 or 0 particles (depending on cluster size) among the remaining 9999 squares. You can do this in (9999^n/n!) ways. As it happens, the first term, with n=5, completely dominates the resulting sum, so we can ignore the others; since 9999 is roughly 10^4 and 5! is roughly 10^2, the number is 10^4 * (10^4)^5 / 10^2 or about 10^22 ways of forming a five-particle cluster.

Now, what of the uniform states? We might start by saying that any state where no square contains two particles is uniform; but this is not very satisfactory, for it allows the particles to be all in neighbouring squares, which we ought to call a cluster; or there could be two groups each of five neighbours, and so on. So, I shall impose an additional constraint: I superimpose a coarser grid on my 100×100, making a hundred large squares, each consisting of ten by ten small squares. I then require that each large square have at most one particle; this allows groups of at most four neighbours (where the corners of four large squares meet), which is still a bit clustered, but better than what we had before. But even now we can form a favoured region of the universe; for example, we could choose a (3×3+1) group of large squares to put our particles in, and we should have to call that a cluster by comparison with the surrounding emptiness, even if the particles are pretty well separated compared to several of them being in one box, as above. So yet another constraint is needed: I make a still coarser grid, four huge squares of 50×50 small squares (five by five large squares), and require each of these to have at least 2 particles. In fact I can *still* make clusters by choosing large squares near the inner corners of the huge squares, but already the calculation is quite complex, so I’ll stop there. Now. For each of four huge squares, choose two from twenty-five large squares; that’s (25*24/2)^4, total 8.1e9; for each of those eight, there are 100 options, so multiply by 10^16 for 8.1e25. Finally add the last two particles in any of 9200 and 9100 small squares, total roughly 6e33. Which is more than the number of clustered states I found above, which is exactly the opposite of what I set out to prove.

The problem is that my ‘uniform’ states include quite a few clusters of three and four particles separated only by two or three squares; that is, there exist sets of particles with separations much smaller than the average, or in other words, states that a human would point to and say “clustered!” The problem cannot be attacked this way, with constraints on what boxes must be filled – at least not if I’m going to be able to do the calculation of the number of states on a single sheet of paper. A different method is needed, with a better definition of ‘uniform’.

Suppose I look at groups of five-by-five squares. Suppose further that I say that four particles in such a group are a cluster – that is, 40% of the universe’s mass concentrated in 0.25% of its volume. Throwing particles at the grid at random, what is the probability of getting at least one such group? To answer this, I’m going to go away from the counting of microstates above, and turn to the Poisson distribution. Suppose you expect to get *M* events on average; then *P(n;M)* is the probability that you will get *n* events.

For each group of five-by-five squares, the probability that a particle will land in it is 0.25%. For ten particles, then, the expectation value for each group is a fourth of a particle. However, it is necessary to be careful, because there is more than one way of dividing the grid into groups of 25 squares, even under the restriction that they be 5×5. In particular, we can move the origin of the coarser grid, relative to the underlying small squares, and wrap around. This gives 25 possible ways of arranging the groups. This is done to avoid the situation where there are 2 particles in one group, 3 in its neighbour, and there exists a 5×5 square that includes all five, but you don’t call it a cluster because of the intervening border. To get all the possible clusters we have to count all the possible borders, as well.

Now. Poisson tells us that with a mean of 0.25, the probability of getting at least 4 events is (roughly) 1.6e-4. There are, however, 10000 possible 25-square groups. Now, in 10000 trials each with a probability of 0.016%, the probability of at least one success is *79.9%*. In other words, just throwing particles at random, you have *four chances in five* of getting such a cluster as I defined above! This, gentlemen, is entropy; and it is also why our universe has stars instead of undifferentiated clouds of hydrogen.

Nice try but your calculations are wrong. That is because you ignore the gravity law which gathers all the particles together. The matter particles in physics are not static “bomb hits” but dynamic ones that keep approaching one another. So the reason that our universe has stars is not because of the statistical probability of this state but because of the gravity law.

Yes, yes, obviously you need gravity to form the stars out of local clusters; but a perfectly even initial distribution would give you one huge black hole, not the clusters-of-clusters distribution that we actually see.

Frankly, I doubt how valid the microstate approach is. During the coalescence of hydrogen gas, the overall volume decreases. And you are telling me that the microstates do not change while the volume decreases?

A classic textbook example of entropy is this: two flasks, one full of hydrogen, the other a vacuum, are connected yet separated by a valve. When the valve is opened, the hydrogen gas flows spontaneously to the other flask.

Why? Because entropy increases during the expansion process. That is what every textbook says. And some proves this by using dots and cells approach, similar to what you did above.

Now, in outer space, the reverse process happens and entropy also increases? I find it extremely funny that the same dots and cells approach is used to “prove” that entropy increases during the coalescence process.

In your example of the valve, the available volume is changed. That’s not the case in the star-cluster example. The situations are not equivalent and the same approach will therefore produce different results. I suggest that you should learn to spot obvious stuff like this before engaging in serious discussion of physics. I also suggest that you might want to say what your alternative theory of this entropy is, and what it predicts.

I love maths stuff and this is a great mathematical analogy to explain the difference between percieved order/disorder in terms of distribution and real uniformity/randomness (if they even exist!).

However, I have to say that although maths is great and is the language of science, the heart of science must always be observation and hard data to back up the maths. Unfortunately this makes it bloody hard to prove anything for certain on either micro- or macroscopic scales. In relation to this argument, without knowing the volume of the universe (if you do know, please tell me ;)!) itself and many other core pieces information on such an enormous and literally unimaginable scale, it’s impossible to really resolve this argument. It’s the same as with life and it’s apparent negative entropy, which again is really fun to discuss.

Anyway, fun maths read and don’t stop thinking about the stuff that really matters!