Benchmarks & Metrics
Randomized Benchmarking
A method for measuring the average error of quantum gates by running long random sequences and watching how fast fidelity decays.
Definition
Randomized benchmarking is the standard way to measure how good a quantum gate is. You apply a long, random sequence of gates that should, in the ideal case, return the qubit to where it started, and then check how often it actually does. By repeating with sequences of different lengths, you see how quickly the success rate drops, and the speed of decay tells you the average error per gate, in a way that is insensitive to errors in preparing and measuring the qubit.
In depth
The key advantage of randomized benchmarking is robustness to SPAM errors. A direct test, such as applying a single gate and measuring, mixes the gate's error with errors in state preparation and readout. By measuring the decay with sequence length, the constant offsets from SPAM go into the fit parameters A and B and drop out of the error rate p, which depends only on the gates. The method was developed by Emerson and colleagues in 2005, refined by Knill and coauthors in 2008, and made into the standard form by Magesan, Gambetta, and Emerson in 2011.
Variants address different needs. Interleaved randomized benchmarking inserts a specific gate between random Cliffords to measure its fidelity, and simultaneous randomized benchmarking runs sequences on several qubits at once to expose crosstalk. Cycle benchmarking and direct randomized benchmarking extend the idea to larger systems and non-Clifford gates. For error-correction experiments, logical randomized benchmarking applies the protocol to logical qubits.
The numbers reported from this method are the headline figures in hardware announcements. Superconducting single-qubit gates have errors around 10 to the minus 4 to 10 to the minus 3, and two-qubit errors around 10 to the minus 3 to 5 times 10 to the minus 3. Trapped ions have shown two-qubit errors near 10 to the minus 4 and single-qubit errors below 10 to the minus 6 in laboratory records. A caveat is that randomized benchmarking measures an average over random sequences, which can hide coherent errors that accumulate in structured circuits.
Technical Definition
In Clifford randomized benchmarking, random Clifford sequences of length m are followed by a single inversion Clifford. The survival probability is fit to F(m) = A p^m + B, where p is the depolarizing parameter and A and B absorb state preparation and measurement errors. The average error per Clifford is r = (1 minus p) times (d minus 1) over d, with d equal to 2 to the n for n qubits.
Visual Explanation: An Analogy
Imagine testing a photocopier by copying a page, then copying the copy, and so on many times. The page gets blurrier with each generation, and how quickly it degrades tells you how good the machine is, regardless of whether your first original was slightly smudged.
Key facts
| Item | Detail |
|---|---|
| Fit model | F(m) equals A times p to the m plus B |
| Output | Average error per Clifford, r equals (1 minus p)(d minus 1) over d |
| Robust to | State preparation and measurement errors |
| Variants | Interleaved, simultaneous, cycle benchmarking, logical randomized benchmarking |
| Limitation | Averages over random sequences and can underestimate coherent errors |
Where the idea comes from
Randomized benchmarking was proposed in 2005 by Emerson, Alicki, and Zyczkowski, and developed for experimental use by Knill and colleagues at NIST in 2008 and by the Gambetta group at IBM and Yale. The 2011 and 2012 papers by Magesan and coauthors established the modern protocol and interleaved variant.
Real-World Use Cases
- Reporting gate fidelities for hardware comparisons.
- Calibrating gates by tracking the effect of parameter changes on the decay.
- Detecting crosstalk with simultaneous benchmarking.
- Benchmarking logical gates in error-corrected experiments.
How it differs from related ideas
- Quantum process tomography reconstructs a full model of a gate but is sensitive to SPAM and scales poorly, while randomized benchmarking gives one robust number.
- Gate set tomography is more complete and also self-consistent, but more demanding than randomized benchmarking.
- Quantum volume and cross-entropy benchmarking test whole circuits, while randomized benchmarking tests gates.
Common Misconceptions
- A good randomized benchmarking number does not guarantee that a specific algorithm will run well.
- It measures average error over random Cliffords, not the error of every possible gate.
- The reported fidelity is not the same as the error in a full-chip circuit with crosstalk.
Frequently asked questions
What does randomized benchmarking measure?
The average error per gate, by fitting how quickly the success probability of long random gate sequences decays.
Why is it robust to measurement errors?
Because state preparation and measurement errors only change the constants in the decay fit, not the decay rate that gives the gate error.
What is interleaved benchmarking?
A variant that inserts a particular gate into the random sequences to measure its individual fidelity.
Related Terms
Keep exploring
- Algorithm: Quantum Error Mitigation Techniques
- Hardware: IBM Heron (r1, r2, and r3)
- Hardware: Quantinuum H2