Dr Pournelle asked an interesting question the other day. (His blog uses the technology of 1995, making permalinks difficult, hence the link to his front page.) The question is this: Entropy always increases. Current cosmological theory has the early universe going from an undifferentiated mass of hydrogen with a bit of helium, to stars and galaxies. What accounts for this apparent decrease in entropy?
The answer is that it’s not a decrease at all, but it does require a bit of digging to see why, hence Dr Pournelle’s confusion. (His post makes it clear that he knows this, he just wants to see the problem worked out.) As usual, the definition of entropy as ‘disorder’ is at the root of the problem. It is by no means unreasonable to argue that stars and galaxies are more orderly than a universe-wide gas. Nonetheless, their entropy is in fact larger, but it is not trivial to show it. Begin by recalling the definition of entropy:
S = kB ln Ω
where kB is Boltzmann’s constant and not important to the argument, and Ω is the number of distinguishable microstates that the system could be in without changing its macrostate. A microstate is a description in terms of positions and momenta of particles; a macrostate is a description in terms of temperature, volume, or other thermodynamic quantities.
First, I will restate the question in its simplest terms: Given a number of particles, is the entropy higher when they are evenly distributed, or when they are clustered? (That is, clouds of gas, or stars.) As a visualisation, consider the universe as a simple two-dimensional grid, with evenly sized squares; and let us distribute particles into the squares, so that the universe can be specified as “1 particle in square 1, no particles in square 2”, and so on. (The argument is precisely the same for three dimensions and boxes, but two dimensions are easier to visualise.) To have some concrete numbers, let us say there are 100 by 100 squares, total 10000, and 10 particles.
(As an aside, the British staff faced a similar problem in WWII. V-2 weapons were hitting London daily; the question was, were the Germans able to aim the rockets, which would make it a good idea to evacuate sensitive targets like the Houses of Parliament; or were they hitting at random, in which case Parliament could take its chances like everyone else? Some of the planners felt that the existence of clusters (that is, multiple hits in single grid locations, with the grid superimposed on a map of London) showed aiming, but a mathematical analysis like the one I’m about to undertake demonstrated that there were no more clusters than you would expect by chance.)
The first problem is that ‘clustered’ and ‘uniformly distributed’ are not so easily defined, as will become clear. Let’s say that anything with five particles or more in one square is a clustered state. This is not the same as saying that anything which fails to meet this condition is uniform – there are grey areas. However, having half or more of the universe’s mass in one location is most certainly a cluster. How many such states exist? First, choose the square with the cluster; ten thousand options. Next, distribute the remaining n=5, 4, 3, 2, 1 or 0 particles (depending on cluster size) among the remaining 9999 squares. You can do this in (9999^n/n!) ways. As it happens, the first term, with n=5, completely dominates the resulting sum, so we can ignore the others; since 9999 is roughly 10^4 and 5! is roughly 10^2, the number is 10^4 * (10^4)^5 / 10^2 or about 10^22 ways of forming a five-particle cluster.
Now, what of the uniform states? We might start by saying that any state where no square contains two particles is uniform; but this is not very satisfactory, for it allows the particles to be all in neighbouring squares, which we ought to call a cluster; or there could be two groups each of five neighbours, and so on. So, I shall impose an additional constraint: I superimpose a coarser grid on my 100×100, making a hundred large squares, each consisting of ten by ten small squares. I then require that each large square have at most one particle; this allows groups of at most four neighbours (where the corners of four large squares meet), which is still a bit clustered, but better than what we had before. But even now we can form a favoured region of the universe; for example, we could choose a (3×3+1) group of large squares to put our particles in, and we should have to call that a cluster by comparison with the surrounding emptiness, even if the particles are pretty well separated compared to several of them being in one box, as above. So yet another constraint is needed: I make a still coarser grid, four huge squares of 50×50 small squares (five by five large squares), and require each of these to have at least 2 particles. In fact I can still make clusters by choosing large squares near the inner corners of the huge squares, but already the calculation is quite complex, so I’ll stop there. Now. For each of four huge squares, choose two from twenty-five large squares; that’s (25*24/2)^4, total 8.1e9; for each of those eight, there are 100 options, so multiply by 10^16 for 8.1e25. Finally add the last two particles in any of 9200 and 9100 small squares, total roughly 6e33. Which is more than the number of clustered states I found above, which is exactly the opposite of what I set out to prove.
The problem is that my ‘uniform’ states include quite a few clusters of three and four particles separated only by two or three squares; that is, there exist sets of particles with separations much smaller than the average, or in other words, states that a human would point to and say “clustered!” The problem cannot be attacked this way, with constraints on what boxes must be filled – at least not if I’m going to be able to do the calculation of the number of states on a single sheet of paper. A different method is needed, with a better definition of ‘uniform’.
Suppose I look at groups of five-by-five squares. Suppose further that I say that four particles in such a group are a cluster – that is, 40% of the universe’s mass concentrated in 0.25% of its volume. Throwing particles at the grid at random, what is the probability of getting at least one such group? To answer this, I’m going to go away from the counting of microstates above, and turn to the Poisson distribution. Suppose you expect to get M events on average; then P(n;M) is the probability that you will get n events.
For each group of five-by-five squares, the probability that a particle will land in it is 0.25%. For ten particles, then, the expectation value for each group is a fourth of a particle. However, it is necessary to be careful, because there is more than one way of dividing the grid into groups of 25 squares, even under the restriction that they be 5×5. In particular, we can move the origin of the coarser grid, relative to the underlying small squares, and wrap around. This gives 25 possible ways of arranging the groups. This is done to avoid the situation where there are 2 particles in one group, 3 in its neighbour, and there exists a 5×5 square that includes all five, but you don’t call it a cluster because of the intervening border. To get all the possible clusters we have to count all the possible borders, as well.
Now. Poisson tells us that with a mean of 0.25, the probability of getting at least 4 events is (roughly) 1.6e-4. There are, however, 10000 possible 25-square groups. Now, in 10000 trials each with a probability of 0.016%, the probability of at least one success is 79.9%. In other words, just throwing particles at random, you have four chances in five of getting such a cluster as I defined above! This, gentlemen, is entropy; and it is also why our universe has stars instead of undifferentiated clouds of hydrogen.