Simple Algorithm
Quantum Computing · Rydberg State · 26 min

Quantum Volume, XEB & Holistic Benchmarks

Single numbers for whole machines, and their caveats.

How do you summarize a whole quantum computer in one number? Quantum volume and cross-entropy benchmarking both run random circuits and compare the output with a classical simulation. Both rest on a striking fact about random quantum states, the Porter–Thomas distribution, and both stop working exactly where quantum computers become interesting: when the circuits can no longer be simulated.

What you’ll learn

  • Define quantum volume precisely: model circuits, heavy outputs, and the 2/32/3 criterion with its confidence requirement.
  • Derive the ideal heavy-output probability (1+ln⁡2)/2(1+\ln2)/2 from Porter–Thomas statistics, and the fidelity threshold 1/(3ln⁡2)1/(3\ln2).
  • Define linear cross-entropy fidelity, show that it estimates circuit fidelity under depolarizing noise, and connect it to random-circuit-sampling experiments.
  • Explain layer fidelity and volumetric benchmarks, and articulate the caveats of single-number benchmarks.

Quantum volume

Quantum volume (Cross, Bishop, Sheldon, Nation & Gambetta 2019) asks: what is the largest square random circuit, with as many layers as qubits, that a device runs well? It probes gates, connectivity, crosstalk and the compiler all at once.

DerivationThe ideal heavy-output probability
  1. s=N pU(x),Pr⁡[s]≈e−s(N=2n→∞)s = N\,p_U(x),\qquad\Pr[s]\approx e^{-s}\qquad(N = 2^n\to\infty)

    Porter–Thomas statistics: for Haar-random (or sufficiently deep random) circuits, the rescaled output probabilities are exponentially distributed.

  2. ∫0smede−s ds=12  ⟹  smed=ln⁡2\int_0^{s_{\text{med}}}e^{-s}\,ds = \tfrac12\;\Longrightarrow\;s_{\text{med}} = \ln2

    The median output has probability p=ln⁡2/Np = \ln2/N.

  3. hideal=∑x∈HUpU(x)≈N∫ln⁡2∞sN e−s ds=[−(s+1) e−s]ln⁡2∞h_{\text{ideal}} = \sum_{x\in H_U}p_U(x)\approx N\int_{\ln2}^\infty\frac{s}{N}\,e^{-s}\,ds = \Big[-(s+1)\,e^{-s}\Big]_{\ln2}^\infty

    Weight each output by its own probability: the heavy half of the strings carries most of the mass.

  4. hideal=1+ln⁡22≈0.847,huniform=12h_{\text{ideal}} = \frac{1+\ln2}{2}\approx0.847,\qquad h_{\text{uniform}} = \frac12

    A perfect device approaches 0.8470.847; a completely depolarized one, sampling uniformly, gets exactly 1/21/2.

What it takes to pass

Model the noise crudely as global depolarization: the device outputs the ideal distribution with probability ff, the circuit fidelity, and uniformly random strings otherwise.

DerivationThe fidelity threshold
  1. h=f hideal+(1−f) 12h = f\,h_{\text{ideal}}+(1-f)\,\tfrac12

    Heavy-output probability under global depolarization.

  2. h>23  ⟺  f>2/3−1/2hideal−1/2=1/6(ln⁡2)/2=13ln⁡2≈0.48h>\tfrac23\iff f>\frac{2/3-1/2}{h_{\text{ideal}}-1/2} = \frac{1/6}{(\ln2)/2} = \frac{1}{3\ln2}\approx0.48

    The circuit fidelity must exceed about 48%.

  3. f≈(1−ε)N2,N2≈3n⌊n/2⌋f\approx(1-\varepsilon)^{N_2},\qquad N_2\approx3n\lfloor n/2\rfloor

    If two-qubit gate errors dominate: each SU(4)SU(4) block compiles into three CNOTs (or CZs), and there are n⌊n/2⌋n\lfloor n/2\rfloor blocks, before counting any routing SWAPs.

Predict before you look

Assume all-to-all connectivity, perfect single-qubit gates and readout, and two-qubit error ε=0.2%\varepsilon = 0.2\%. Using f≈e−εN2f\approx e^{-\varepsilon N_2} with N2≈3n2/2N_2\approx3n^2/2 and the threshold f>0.48f>0.48 (so εN2<ln⁡(1/0.48)≈0.73\varepsilon N_2<\ln(1/0.48)\approx0.73), what is the largest width nn that passes?

Show the answer

15 qubits

32n2×0.002<0.73\tfrac32n^2\times0.002<0.73 gives n2<244n^2<244, so n≤15n\le15 and VQ=215V_Q = 2^{15} in this idealized model. With exact counts, n=15n = 15 has N2=315N_2 = 315 and f≈0.53f\approx0.53, while n=16n = 16 has N2=384N_2 = 384 and f≈0.46f\approx0.46. Sparse connectivity, crosstalk and the confidence requirement all push the real number down.

The model explains several features of reported quantum volumes. Because N2N_2 grows like n2n^2, each extra qubit of width demands a lower error rate, roughly ε∝1/n2\varepsilon\propto1/n^2. All-to-all connectivity avoids routing overhead, which is one reason trapped-ion machines have reported the largest quantum volumes. And because the heavy set must be computed classically for each circuit, quantum volume can only be certified at widths that can be simulated, a few tens of qubits.

Cross-entropy benchmarking

Cross-entropy benchmarking (XEB) scores samples x1,…,xkx_1,\ldots,x_k from a random circuit by how probable they are under the ideal distribution (Boixo et al. 2018; Arute et al. 2019).

DerivationWhy XEB estimates fidelity
  1. Ex∼q[N pU(x)]−1,q=f pU+(1−f) 1N\mathbb E_{x\sim q}\big[N\,p_U(x)\big]-1,\qquad q = f\,p_U+(1-f)\,\tfrac1N

    Expected score of a globally depolarized device.

  2. =f(N∑xpU(x)2−1)= f\Big(N\sum_xp_U(x)^2-1\Big)

    Using ∑xpU(x)=1\sum_xp_U(x) = 1.

  3. EU[∑xpU(x)2]=2N+1  ⟹  E FXEB=f N−1N+1≈f\mathbb E_U\Big[\sum_xp_U(x)^2\Big] = \frac{2}{N+1}\;\Longrightarrow\;\mathbb E\,F_{\text{XEB}} = f\,\frac{N-1}{N+1}\approx f

    For Haar-random UU each pU(x)p_U(x) has second moment 2/(N(N+1))2/(N(N+1)); random circuits of sufficient depth approximate this.

Predict before you look

A random circuit contains 400 two-qubit gates, each with error 0.5%0.5\%; everything else is perfect. Under the digital error model F≈∏i(1−εi)F\approx\prod_i(1-\varepsilon_i), what XEB fidelity do you expect, in percent?

Show the answer

13.5 %

0.995400=e400ln⁡0.995≈e−2.0≈13.5%0.995^{400} = e^{400\ln0.995}\approx e^{-2.0}\approx13.5\%. Fidelity decays exponentially with the number of gates, which is why XEB values for large experiments are tiny, though still statistically resolvable with millions of samples.

Layer fidelity, volumetric benchmarks, and their limits

Quantum volume and XEB stop scaling when simulation does. Benchmarks for hundreds of qubits avoid simulating the full circuit.

Check your understanding

A 100-qubit chain has 99 two-qubit gates in its layers and a measured layer fidelity of 0.370.37. What is the error per layered gate?

  1. 0.37%0.37\%
  2. 0.64%0.64\%
  3. about 1.0%1.0\%
  4. 63%63\%
Show the answer

C. about 1.0%1.0\%

EPLG=1−0.371/99=1−e(ln⁡0.37)/99≈1−e−0.0100≈1.0%\mathrm{EPLG} = 1-0.37^{1/99} = 1-e^{(\ln0.37)/99}\approx1-e^{-0.0100}\approx1.0\%. Equivalently, a product of 99 factors of about 0.990.99 is about e−1≈0.37e^{-1}\approx0.37.

BenchmarkScales toNeeds classical simulation?Main blind spot
Quantum volumetens of qubitsyesone circuit shape; saturates at simulable sizes
Linear XEBtens of qubits (directly)yesassumes depolarizing-like noise
Layer fidelity / EPLGhundreds of qubitsnospecific layers; a Pauli-twirled average
Mirror-circuit and volumetric benchmarkshundreds of qubitsnothe choice of circuit family
Application benchmarksvariessometimesdepend on compiler and problem instance
Volumetric benchmarking (Blume-Kohout & Young 2020) plots success over a grid of widths and depths for a chosen circuit family instead of reducing it to one number; mirror circuits (Proctor et al. 2022) make success checkable without simulation. Speed metrics, such as circuit layers per second, complement all of these.

Key takeaways

  • Quantum volume is 2n2^n for the largest width nn at which depth-nn random model circuits produce heavy outputs more than 2/32/3 of the time, with confidence.
  • Porter–Thomas statistics give an ideal heavy-output probability (1+ln⁡2)/2≈0.85(1+\ln2)/2\approx0.85; under global depolarization, passing requires circuit fidelity above 1/(3ln⁡2)≈0.481/(3\ln2)\approx0.48.
  • Linear XEB, N⟨pU(xi)⟩−1N\langle p_U(x_i)\rangle-1, estimates circuit fidelity when noise is depolarizing-like; Sycamore's 2019 experiment reported about 0.2%0.2\% at 53 qubits.
  • Layer fidelity and mirror-circuit benchmarks scale to hundreds of qubits without simulation; every single number hides structure and can be optimized for.

Check yourself

In the interactive lesson these are scored with your confidence, and what you miss comes back for review.

Question 1

For an ideal device running large random model circuits, the heavy-output probability approaches

  1. 1/21/2
  2. 2/32/3
  3. 11
  4. (1+ln⁡2)/2≈0.85(1+\ln2)/2\approx0.85
Show the answer

D. (1+ln⁡2)/2≈0.85(1+\ln2)/2\approx0.85

Porter–Thomas statistics give ∫ln⁡2∞s e−s ds=(1+ln⁡2)/2\int_{\ln2}^\infty s\,e^{-s}\,ds = (1+\ln2)/2. The 2/32/3 bar sits between this and the uniform value 1/21/2.

Question 2

A sampler ignores the circuit and outputs uniformly random nn-bit strings. Its expected linear XEB fidelity is

  1. 00
  2. 1/21/2
  3. 1/N1/N
  4. −1-1
Show the answer

A. 00

For uniform xx, E[N pU(x)]=N⋅1N∑xpU(x)=1\mathbb E[N\,p_U(x)] = N\cdot\tfrac1N\sum_xp_U(x) = 1, so FXEB=1−1=0F_{\text{XEB}} = 1-1 = 0.

Question 3

Why can quantum volume not be measured with 100-qubit, depth-100 model circuits?

  1. Because Haar-random SU(4)SU(4) gates cannot be compiled on hardware
  2. Because deciding which outputs are heavy requires classically computing the ideal output distribution
  3. Because the 2/32/3 threshold becomes unreachable at large nn even for an ideal device
  4. Because quantum volume is only defined up to 64 qubits
Show the answer

B. Because deciding which outputs are heavy requires classically computing the ideal output distribution

The heavy set is defined by the median of pUp_U, which must be computed classically for every circuit; that is infeasible far beyond a few tens of qubits.

Question 4

Under a global-depolarizing model, what circuit fidelity ff is needed to pass the heavy-output test at large width?

  1. f>2/3f>2/3
  2. f>1/2f>1/2
  3. f>1/(3ln⁡2)≈0.48f>1/(3\ln2)\approx0.48
  4. f>(1+ln⁡2)/2≈0.85f>(1+\ln2)/2\approx0.85
Show the answer

C. f>1/(3ln⁡2)≈0.48f>1/(3\ln2)\approx0.48

Solve f 1+ln⁡22+(1−f) 12>23f\,\tfrac{1+\ln2}{2}+(1-f)\,\tfrac12>\tfrac23: f>1/6(ln⁡2)/2=13ln⁡2f>\frac{1/6}{(\ln2)/2} = \frac{1}{3\ln2}.

Further reading

  • A. W. Cross, L. S. Bishop, S. Sheldon, P. D. Nation & J. M. Gambetta, "Validating quantum computers using randomized model circuits", Physical Review A 100, 032328 (2019).
  • S. Aaronson & L. Chen, "Complexity-theoretic foundations of quantum supremacy experiments", Proceedings of the Computational Complexity Conference (CCC) (2017).
  • S. Boixo et al., "Characterizing quantum supremacy in near-term devices", Nature Physics 14, 595 (2018); F. Arute et al., "Quantum supremacy using a programmable superconducting processor", Nature 574, 505 (2019).
  • D. C. McKay et al., "Benchmarking quantum processor performance at scale", preprint (2023).
  • R. Blume-Kohout & K. C. Young, "A volumetric framework for quantum computer benchmarks", Quantum 4, 362 (2020).