Quantum Information Theory
Entropy, channels and the ultimate limits of communication.
How many bits does a qubit hold? Infinitely many, if you count the real parameters of its state. Exactly one, if you ask how many you can get back out. Quantum information theory makes this precise with three tools: von Neumann entropy, quantum channels and the Holevo bound. From them it derives the ultimate rates for sending information through noisy quantum channels.
What you’ll learn
- Compute Shannon and von Neumann entropies and state their operational meaning as compression rates.
- Evaluate joint and conditional entropies and mutual information for quantum states, and interpret negative conditional entropy.
- Write quantum channels in Kraus form and compute how depolarizing, dephasing and amplitude-damping noise act on the Bloch vector.
- State the Holevo bound and use it to show that qubits carry at most classical bits.
- Distinguish the classical, quantum and entanglement-assisted capacities, and evaluate all three for the erasure channel.
Shannon's two theorems
Classical information theory rests on one quantity. For a random variable with distribution , the Shannon entropy is
For a biased coin it is the binary entropy , which runs from (certain) to bit (fair).
Shannon (1948) gave entropy two operational meanings. Source coding: independent copies of can be compressed into about bits and recovered with vanishing error, and no fewer bits suffice. Channel coding: a noisy channel carries information reliably at any rate below its capacity
and at no rate above it.
A binary symmetric channel flips each bit with probability . Its capacity is . How many bits per channel use is that?
Show the answer
0.5 bits
, so . Not a coincidence you have seen before: the 11% BB84 threshold is where vanishes, i.e. where .
Von Neumann entropy
The von Neumann entropy of a state with eigenvalues is the Shannon entropy of its spectrum:
- , with iff is pure and iff .
- : entropy is basis-independent.
- Concavity: . Mixing never decreases entropy.
- For a qubit with Bloch vector , the eigenvalues are , so .
Alice flips a fair coin and sends or .
Trace 1 and determinant . Equivalently, the Bloch vector is , of length .
The overlap hides part of the coin's randomness. In general , with equality iff the are orthogonal.
Entropy of composite systems
For a bipartite state with marginals , define the conditional entropy and mutual information
Classically : knowing more cannot leave you more uncertain about than about the whole. Quantum mechanically this fails.
| State of | |||||
|---|---|---|---|---|---|
| (product) | 0 | 0 | 0 | 0 | 0 |
| (classically correlated) | 1 | 1 | 1 | 0 | 1 |
| (Bell state) | 1 | 1 | 0 | 2 |
A two-qubit state has . What can you conclude?
- The state is mixed
- The state is entangled
- Subadditivity is violated
- Nothing: conditional entropy can be negative for classically correlated states too
Show the answer
B. The state is entangled
Every separable state has , so negative conditional entropy is a witness of entanglement. Operationally, is the entanglement Alice and Bob gain when Alice transfers her share to Bob in quantum state merging (Horodecki, Oppenheim & Winter, 2005).
Quantum channels
A channel is the most general physical process taking states of to states of . Mathematically it is a linear, completely positive, trace-preserving (CPTP) map, and every such map has a Kraus representation
Equivalently (Stinespring), is a unitary interaction with an environment that starts in a fixed state, followed by discarding : . Noise is information leaking into an environment you do not control.
| Channel | Kraus operators | Bloch vector |
|---|---|---|
| Bit flip | ||
| Phase flip (dephasing) | ||
| Depolarizing | ||
| Amplitude damping |
Multiply out. moves population from to ; damps the coherences.
Using .
The coherences shrink by . With this is decay, and the fixed point is .
Which of these maps on a qubit is not a valid quantum channel?
- (replace any input by the maximally mixed state)
- (reset to )
- (transpose in the computational basis)
- (complete dephasing)
Show the answer
C. (transpose in the computational basis)
The transpose maps every density matrix to a density matrix, so it is positive, but it is not completely positive: applied to one half of it produces an operator with eigenvalue . A physical process must stay valid when it acts on part of an entangled system.
The Holevo bound and channel capacities
The quantity is the Holevo information of the ensemble. Return to the source: its states are pure, so bits. The best single-copy measurement does even worse: measuring in the basis symmetric about the two states errs with probability , giving bits, which for two equiprobable pure states is known to be optimal. The maximum over measurements, the accessible information, can be strictly below .
Superdense coding sends 2 bits per transmitted qubit only because the receiver already holds the other half of a Bell pair: the Holevo bound then applies to the two-qubit system, allowing 2 bits.
| Capacity | Task | Formula (key theorem) | Erasure channel, erasure prob. |
|---|---|---|---|
| , classical | Send bits through | Regularized Holevo information (Holevo–Schumacher–Westmoreland) | |
| , quantum | Send qubits (or create entanglement) | Regularized coherent information (Lloyd–Shor–Devetak) | |
| , entanglement-assisted | Send bits with unlimited shared entanglement | Max mutual information (Bennett–Shor–Smolin–Thapliyal) |
Key takeaways
- Von Neumann entropy is the Shannon entropy of the spectrum and the optimal rate of quantum data compression (Schumacher).
- Quantum conditional entropy can be negative, for a Bell pair, which certifies entanglement; strong subadditivity underlies almost every theorem.
- Channels are CPTP maps, with , equivalently a unitary on system plus environment followed by a partial trace.
- Holevo bound: , so qubits carry at most bits.
- A quantum channel has several capacities (, , ); for the erasure channel they are , and .
Check yourself
In the interactive lesson these are scored with your confidence, and what you miss comes back for review.
For the Bell state , what are , and ?
Show the answer
C.
The global state is pure, so , while each marginal is , so . Then , impossible for classical variables.
The depolarizing channel with acts on . What are the eigenvalues of the output?
Show the answer
B.
The Bloch vector shrinks from length 1 to , so the eigenvalues are . Directly: .
Alice encodes one of three equiprobable messages in the "trine": three pure qubit states apart on a great circle of the Bloch sphere. How much classical information can Bob extract per qubit?
- bits, one trit, with the right measurement
- Exactly 0, because the states are not orthogonal
- 2 bits, using a POVM with three outcomes
- At most 1 bit, by the Holevo bound
Show the answer
D. At most 1 bit, by the Holevo bound
The average state is , so bit, and no measurement can beat it. The optimal measurement actually extracts only bits.
What state does repeated amplitude damping drive any qubit toward?
- The maximally mixed state
- It leaves the input unchanged up to a phase
Show the answer
A.
Under the fixed point is , and the coherences shrink by each time. Every state decays to , the ground state: this is energy relaxation, .
For a quantum erasure channel with erasure probability , which statement is correct?
- and
- , because most transmissions are erased
- , yet and
Show the answer
C. , yet and
For the quantum capacity vanishes (no-cloning), but classical bits still get through at , and shared entanglement doubles that to .
Go deeper
- Quantum Channels: Kraus & Stinespring · CPTP maps, the Choi isomorphism and dilations.
- Von Neumann Entropy & the Holevo Bound · Why n qubits carry at most n classical bits.
- Channel Capacities & Coding Theorems · Classical, quantum and entanglement-assisted capacities.
Further reading
- M. A. Nielsen & I. L. Chuang, Quantum Computation and Quantum Information, Chapters 8, 11 and 12 (Cambridge, 10th anniv. ed., 2010).
- M. M. Wilde, Quantum Information Theory, 2nd ed. (Cambridge, 2017).
- J. Preskill, Lecture Notes for Physics 219: Quantum Computation, chapter on quantum Shannon theory (Caltech).
- C. E. Shannon, "A mathematical theory of communication", Bell System Technical Journal 27, 379 and 623 (1948).