The Von Neumann Entropy: a Reply to Shenker
Leah
Henderson
Abstract
Shenker has
claimed that Von Neumann’s argument for identifying the quantum mechanical
entropy with the Von Neumann entropy, 
, is invalid. Her claim rests on a misunderstanding of the
idea of a quantum mechanical pure state. I demonstrate this, and provide a
further explanation of Von Neumann’s argument.
The quantum mechanical entropy is generally taken to be the Von
Neumann entropy . Orly Shenker ([1999]) has claimed that Von Neumann’s
argument in support of this choice is not valid. I show here that Shenker is
wrong and explain how Von Neumann’s argument works in a full quantum mechanical
treatment.
Identifying the quantum mechanical entropy is an issue of
considerable importance for quantum statistical mechanics. Von Neumann ([1955])
uses it to argue that quantum mechanical measurement is irreversible. He is
also able to show that the constraints imposed by quantum mechanics are closely
related to those of thermodynamics in the sense that the ability to distinguish
non-orthogonal quantum states would be equivalent to a violation of the second
law being possible[1]. In
quantum information theory Von Neumann entropy appears as the analogue of the
The first point that Von Neumann establishes is that all pure
quantum states have the same entropy and that this may be taken to be zero[2].
His problem is then to find the entropy of a mixed state, which may arise as
the result of a measurement on a pure state. He argues that a mixture of
quantum states may be treated in close analogy with a mixture of classical
ideal gases. A mixture of classical ideal gases occupying a volume 
may be separated using
semi-permeable membranes which are permeable to one gas but opaque to the
others. The separation may be done reversibly using the membranes as pistons
and keeping the whole system in contact with a reservoir at constant
temperature[3]. To show
that mixing has a thermodynamic effect, consider the following process
restoring the mixture. After separation, each separated gas occupies the
original volume alone. To return to
the mixture, each gas is compressed to a volume 
(where 
is the concentration
of the 
th gas). The compression requires
work 
to be invested, and
the entropy of the gas is reduced by 
. An increase in entropy of the same amount must then be
associated with the mixing step of removing the partitions. This is the ‘mixing
entropy’.
Von Neumann considers an ‘ideal gas’ of quantum states which may again be separated by semi-permeable membranes if and only if they are orthogonal[4]. The ‘mixing entropy’ for the ensemble of eigenstates of the mixed state would then be given by the Von Neumann entropy. The problem of ‘whether it is legitimate to treat quantum states in the same way as classical ideal gases’ is addressed by Peres ([1990]). Here I assume this starting point, as does Shenker in her critique.
Shenker takes the simple example of the mixture 
. The states 
and 
may be spin
eigenstates of a spin-half particle. Then Von Neumann’s argument runs as
follows. In the initial state, we have a gas of particles each in the pure state 
, occupying a box of volume . A measurement is then made in the 
basis, giving an
equally weighted mixture of particles in states and . Semi-permeable partitions are used to separate the states,
resulting in one box of volume occupied by particles
in state and one box of volume occupied by particles in state . Each gas is then compressed to a volume 
, to restore the original total volume . Then particles in each box are returned to the state 
, the partitions are removed and the initial state is
recovered. We have a complete cycle.
Now consider the entropy changes. We will compare what we expect
thermodynamically to what we calculate using the Von Neumann entropy. At this
stage the Von Neumann entropy may be regarded as just some function of the state 
, which may or may not behave like thermodynamic entropy.
The use of semi-permeable partitions requires no net work and hence
involves no thermodynamic entropy change. The step which requires investment of
work is the compression of the two gases to half their volume. The entropy of
the system decreases in this process. There is no change in entropy in
restoring the initial state because, as Von
Neumann shows, the transformation can be achieved reversibly, and removal of
the partitions has no thermodynamic effect. Since the cycle is closed, the
total change in entropy is zero. The entropy has decreased in the compression
step, so it must have increased at some other point. By a process of
elimination, it must have increased in the initial measurement step. This is
Von Neumann’s argument for the irreversibility of measurement.
Now are these thermodynamic changes in entropy reflected in the behaviour of the Von Neumann entropy? Shenker claims that they are not, but this is because she has not calculated them correctly. I now show how they should be calculated.
First notice that because the particles are taken
to be distinguishable and non-interacting, the state of the gas of particles
throughout is the tensor product of the states of the individual particles. The Von Neumann entropy is additive[5],
i.e. 
, therefore in calculating entropy changes we need only
consider one particle.
It is important to consider not only the internal, or spin, degrees
of freedom of the particle, but also its spatial degrees of freedom. The
measuring apparatus must also be taken into account. The initial state of the
system is given by 
, where 
is the thermalised
state of the spatial degrees of freedom for a free particle confined in a box
of volume . I will assume for simplicity that there is no interaction
between the spin and spatial degrees of freedom except during the separation
stage. The initial state is pure, so the Von Neumann entropy of the spin part
is zero. The Von Neumann entropy of 
is 
, where 
is a constant
depending on temperature[6].
plays
no role here, since the processes are isothermal. I take it to be zero for
simplicity, since we are only interested in entropy changes. I also take the
initial volume 
. The Von Neumann entropy of the initial state of the system
is then zero.
The first step is a measurement in the 
basis. There is no
change in the volume of the system. The entropy change is due entirely to the
spin degrees of freedom. After the measurement, the spin degrees of freedom are
in the mixed state 
and so, taking the
logarithm with base two, 
. The entropy has increased as expected.
The second step is separation of the gases.
After this, the system is still in the state 
and so there is no
change of Von Neumann entropy, as expected.
Shenker claims that the state of the system
at this stage is pure, and so the Von Neumann entropy must go back to zero. She
then argues that the thermodynamic entropy and the Von Neumann entropy cannot
be identified because at this stage they behave differently. The idea that the
state is pure arises from thinking that the system has been collapsed by the
measurement into either state or state . However Shenker only claims the state is pure after the
reversible separation into two separate boxes has been carried out. Immediately
after the measurement in the first step everyone agrees that the state of the
system is mixed. The effect of the measurement may be ontologically to convert
the system into a state of spin-up or spin-down[7],
but this does not mean we write the state of the system as pure at this stage,
because we lack knowledge of which
pure state the system is in. What can have changed after the separation? The
only thing that may have changed is that the particle may now be considered to
be ‘labelled’ by which box it is in. If we were to consult this label, surely
we would have removed the classical ignorance of the state and so we should
write down a pure state?
To think in this way is to misunderstand the
meaning of a quantum mixture. There are two standard ways of thinking of a
mixed state. The first is to do with incomplete knowledge of the state. The
state of a quantum system is supposed to contain all the information that is known
about it. This means that a state cannot be written as pure if there is a
classical probability distribution over the different possible states the
system may be in. In our situation, the preparation of the state is given by
the measurement step at stage one and the separation at stage two. This
preparation produces the pure states and with equal
probabilities. In a particular trial the observer may take note of the
measurement result, and he therefore discovers that he has say a . If he applies a projective measurement in the basis, he could
predict that he will measure . However this does not mean that if someone handed him
another state prepared in the same way, that he could again predict that the
outcome of his measurement would be . In this sense the observer does not know the state of the
system which is being prepared, and it is because of this ignorance that the
state is mixed. Looking at the measurement result does not remove the fact that
there is a probability distribution over the possible outcomes.
The second way of looking at mixtures is to
say they arise from correlations to an environment. From this point of view,
‘taking note of the measurement result’ has a physical meaning. It means
converting the correlation to the measuring apparatus from an entanglement 
to a classical correlation between the system states and the
distinct orthogonal (and hence distinguishable) states of the measuring
apparatus. The state is correlated with the
measurement outcome 
, corresponding to the final state 
and similarly for
. The state of the system and measuring device is now a
mixture of these two outcomes: 
. The system and measuring apparatus are no longer entangled,
but there is a classical correlation between them. Tracing out the measuring
apparatus, the state of the spin part of the system itself remains in the mixed
state . The Von Neumann entropy is one.
To complete the cycle of entropy changes, the next step is
compression. The spin degrees of freedom are not affected by this. The change
in Von Neumann entropy of the spatial degrees of freedom is now
. The Von Neumann entropy of the gas decreases, as expected,
and returns to zero. The pure state is restored by disentangling
the system from the measurement apparatus. This may be done reversibly, either
by the right choice of unitary in each side of the box, or by the measurement
method suggested by Von Neumann[8].
There is no change in the Von Neumann entropy. Removal of the partitions
produces no change either.
We see then that, contrary to Shenker’s claim, the Von Neumann entropy function exactly mirrors the expected thermodynamic changes. This is a convincing argument for taking it as the quantum mechanical entropy, and this is what Von Neumann does.
Acknowledgements
I thank J. Ladyman, A. Peres, S. Popescu, O. Shenker and C. Timpson for helpful discussions. I thank anonymous referees of the journal for useful criticisms.
References
Peres, A.
[1990]: ‘Thermodynamic Constraints on Quantum Axioms’, in: W. H. Zurek, (ed.) Complexity,
Entropy and the Physics of Information,
Peres, A.
[1995]: Quantum Theory: Concepts and
Methods,
Schumacher B. [1995]: ‘Quantum Coding’, Physical Review A, 51, pp.2738-47.
Shenker, O. R.
[1999]: ‘Is 
the Entropy in Quantum
Mechanics?’, British
Journal for the Philosophy of Science, 50,
pp. 33-48.
Von Neumann, J.
[1955]: The Mathematical Foundations of
Quantum Mechanics, trans. R. T. Beyer,
Wehrl, A. [1978]: ‘General Properties of Entropy’, Reviews of Modern Physics, 50, pp.221-260.
Zemanksy, M. W.
[1951]: Heat and Thermodynamics,
Department of Mathematics
University Walk
e-mail: l.henderson@bristol.ac.uk