Diagnostics: Explain a Failure · 25 min · free lesson

Measure the Tail, Not Just the Average

Compute an explicitly defined latency percentile.

Core idea

State the percentile convention and preserve the original measurements.

Diagram for Measure the Tail, Not Just the Average: a Rust code example flows to the lesson rule and the key observation.
The worked implementation exercises compute an explicitly defined latency percentile. Its tests demonstrate the stated finite contract.

Understand it

Averages can hide slow requests. A service with many quick responses and a few multi-second waits may have an acceptable mean while some users regularly time out. Record a latency distribution and inspect tail behavior alongside throughput and error rate.

This exercise defines p95 using the nearest-rank rule: sort a copy of the positive or zero millisecond observations, choose rank ceil(0.95*n), and return that element. Return None for an empty sample. Do not mutate the caller's measurements or invent interpolated values. A percentile from a small sample has limited statistical precision; the exact calculation is not evidence that the load generator or measurement conditions were realistic. The desktop companion keeps raw samples, concurrency and toolchain identity with a before/after report.

The route project is Task Ledger: first keep a reproducible source history, then expose its data through a CLI and HTTP boundary, persist it in SQL, diagnose failures and prepare a release. This lesson owns one contract in that project. Run the worked example before changing the starter and explain which input makes the tempting shortcut fail.

Terminal practice: Run python3 scripts/latency.py in the desktop companion, retain its raw samples and error count, then compare p95 before and after a change.

The compiler lab is independent of the reading question. Edit solution.rs; the app supplies a separate caller with normal, empty and adversarial inputs. A successful unchanged demonstration does not prove your implementation. After the lab, apply the same contract to the route companion project. Its README separates the on-device adapter from the desktop Cargo/TCP profile.

Read the code

pub fn p95(samples: &[u64]) -> Option<u64> {
    if samples.is_empty() {
        return None;
    }
    let mut sorted = samples.to_vec();
    sorted.sort_unstable();
    let n = sorted.len();
    let rank = n - n / 20;
    Some(sorted[rank - 1])
}

fn main() {
    assert_eq!(p95(&[1, 2, 3, 100]), Some(100), "case 1");
    assert_eq!(p95(&[]), None, "case 2");
    assert_eq!(p95(&[7]), Some(7), "case 3");
    assert_eq!(
        p95(&(1..=20).collect::<Vec<u64>>()),
        Some(19),
        "case 4"
    );
    assert_eq!(p95(&[9, 1, 9, 0]), Some(9), "case 5");
    println!("worked example passed");
}

The worked implementation exercises compute an explicitly defined latency percentile. Its tests demonstrate the stated finite contract.

Practice starter

pub fn p95(samples: &[u64]) -> Option<u64> {
    todo!("implement the contract")
}

Open local example in Playground ↗

Try it

Return nearest-rank p95: sort a copy and choose rank ceil(0.95*n), using one-based ranks. Empty input returns None. Retain exact u64 observations. Keep the public signature. Open the compiler lab and edit solution.rs; do not edit the private verification caller.

What success looks like: The separate caller accepts the changed solution and rejects unchanged TODO, a wrong implementation and a printed success message.

Check your understanding

Why can a good average coexist with a poor user experience?

Reveal the answer
  1. Percentiles eliminate the need for sample context
  2. Slow tail requests can be hidden by many fast ones
  3. The average is always the slowest request

A mean alone does not describe the slow tail or the measurement conditions.

Go deeper

Common misconception: The average is always the slowest request

State the percentile convention and preserve the original measurements.

  1. Identify the boundary: Compute an explicitly defined latency percentile. Write down its accepted inputs and observable output.
  2. Exercise the failure: A mean alone does not describe the slow tail or the measurement conditions.
  3. Check the independent caller: Return nearest-rank p95: sort a copy and choose rank ceil(0.95*n), using one-based ranks. Empty input returns None. Retain exact u64 observations.

Transfer challenge: Apply this contract to Task Ledger, introduce the failure described in the lesson, and show the failing check before fixing it.

Connections

  • Propagate One Request Identity: Validate a bounded correlation ID
  • Bound the Retry Budget: Retry transient failures within attempt and time limits