The consumer sleeps, the producer pays

New to the Disruptor? An open-source library at the core of the LMAX exchange, moving events between threads through a pre-allocated ring buffer. A producer publishes into the ring; each consumer reads at its own pace. Latencies are quoted as a median and a p99.9 — the typical call and the slowest in a thousand — because averages hide exactly the behaviour that pages you. Every [src] link opens the code at a87bf42.

Most of the trading day, a consumer that has caught up to the producer has nothing to do. Then the open hits and the first order lands on a thread that has been idle — the one deciding whether a fill is acknowledged in time. What it did while idle set the price of that moment, and there's no setting where it costs nothing.

Spin: buy the tail with a core

BusySpinWaitStrategy is a bare loop (src):

while ((availableSequence = dependentSequence.get()) < sequence)
{
    barrier.checkAlert();
    ThreadHints.onSpinWait();
}

It reads the sequence, and if the message isn't there it reads again. onSpinWait() isn't a yield — it's a hint to the CPU that says "I'm spinning, don't fight me on the pipeline." The thread never sleeps and never gives the core back. So when the message lands there's no wakeup: the consumer is already looking.

The price is a core at 100%, forever, whether one message a second is flowing or a million.

Block: sell the core back, pay at the wakeup

BlockingWaitStrategy owns a lock and a condition, and actually sleeps (src — the method then falls through to a short catch-up spin, at :56-62):

long availableSequence;
if (cursorSequence.get() < sequence)
{
    lock.lock();
    try
    {
        while (cursorSequence.get() < sequence)
        {
            barrier.checkAlert();
            processorNotifyCondition.await();
        }
    }
    finally
    {
        lock.unlock();
    }
}

Read the direction of that test carefully. cursorSequence is the last sequence the producer published, so cursorSequence.get() < sequence means the message hasn't arrived yet — the lock is taken when the consumer has caught up and has nothing left to do, not when it's behind. A consumer with a backlog never sleeps at all.

await() parks the thread and the core goes back to the OS. That's the whole appeal. But now a message arriving isn't enough — the sleeping thread has to be woken, and being woken is a scheduler round-trip: the producer signals, the OS marks the thread runnable, a core has to come free. That path is measured in microseconds and it's jittery, and the jitter lands exactly where you can't afford it.

What that costs

Five of the eight strategies the library ships, on a dedicated Xeon — 21 of its 22 cores given to the benchmark, the 22nd held by a spinner that keeps the kernel's statistics watchdog off them. The producer requested a 50 µs park between messages; timer slack put the achieved gap at ~100 µs — long enough that the parking strategies actually park:

strategy          p50(ns)   p99.9(ns)   consumer CPU
BUSY_SPIN             163         230        100%
PHASED_BACKOFF        156         231        100%
YIELDING              428         771        100%
SLEEPING           24,543      53,547       60.1%
BLOCKING            4,231      20,891        3.2%

Spin holds a flat, sub-microsecond tail and pays a whole core. Block hands it back — under 4%, essentially idle — and pays at the wakeup: 4.2 µs median, 20.9 µs at p99.9.

Notice what that figure is and isn't. It's the cost of a wakeup, charged whenever the consumer had gone to sleep — by the fast path above, whenever it had caught up. That is the regime the harness runs. It does not measure a burst, so what happens to a consumer already behind is read from the code, not a run. And read 4.2 µs as the typical price; 20.9 µs is the slowest call in a thousand.

One caveat on the 230 ns: it's the least stable number here — five iterations spanning 206–454 ns, so the ratio against blocking runs from 101× down to 46×.

The middle of that table isn't a middle. Yielding is nearly as quick as spinning and still burns 100% of a core — Thread.yield() on an unloaded box has nothing to yield to. Sleeping loses to blocking on both axes here — though neither middle strategy pays the next section's producer-side bill: both signal methods are empty (src, src) — read from the code, not measured. And PhasedBackoff (src), the library's own designed compromise — spin, then yield, then fall back — posts the fastest row here for a dull reason: our harness pairs it with a one-millisecond spin budget (the library ships no default), and against ~100 µs arrivals it never leaves its spin phase. Busy-spin wearing a different name, at busy-spin's price in the CPU column.

So on a dedicated core the two poles are the real choice, and the middle collapses into one or the other.

The twist: blocking bills the producer

Look at what the producer does on publish (src):

cursor.set(sequence);
waitStrategy.signalAllWhenBlocking();

There's no condition on that second line. Every call to publish() makes it. For a busy-spin consumer it's an empty no-op (src) — free. For a blocking consumer it's a real lock.lock() and signalAll() (src), on the producer's hot path.

How much? A second harness times publish() alone, on the same box:

publish() cost in ns              p50      p99    p99.9
consumer parked between messages
  BUSY_SPIN                        33       34       35
  BLOCKING                      1,438    1,609   18,262
consumer mostly keeping up
  BUSY_SPIN                        20       40       69
  BLOCKING                         39    1,674    3,295

These are un-batched publishes: publish(lo, hi) delegates to publish(hi) (src), so a producer that claims a hundred slots at once pays this bill once, not a hundred times.

With the consumer parked, publishing costs 1.4 µs at the median and 18.3 µs at p99.9, against a flat 33 ns. Something really is being woken. When the consumer is mostly keeping up the median falls to 39 ns — but the producer's tail still reaches 3.3 µs, because a consumer that is "mostly keeping up" still parks whenever it briefly drains the ring, and nobody controls when.

So a consumer's choice to sleep is charged to the producer, which never made it.

And that policy isn't set per consumer. It's fixed once, when the ring is built (src), and the ring hands that same instance to every consumer that attaches (src). You can hand-roll the object a consumer waits on — the interface is public (src), and the processor takes any implementation (src). A hand-rolled spinning consumer measured 166 ns at the median, 590 ns at p99.9 on a ring everyone else blocks on — but it works one way only, and it saves that consumer, not the producer, still publishing into a blocking ring.

The law

No setting makes waiting free; each only relocates the cost — onto a core, onto the tail, onto the producer, and because the library scopes the knob to the ring, you relocate it for every consumer it wires up at once. One ring spinning: every consumer gets the flat tail and every consumer burns a core, including the nightly reporting one that never needed it. One ring blocking: for the consumers gated on the producer the cores come back and the 20.9 µs tail comes with them, and the producer pays on every publish for a choice it never made.

The library's own mitigation makes the point. LiteBlockingWaitStrategy (src) skips the signal when nobody is parked: it saves 11 ns of blocking's 39 where the tax was already cheap, nothing where it hurts, and widens the producer's tail from 3,295 ns to 6,352 doing it.

What to take away

1. Find out whose knob it actually is. The expensive mistake isn't choosing wrong; it's assuming the setting is per-consumer when the library scopes it per ring. Check a setting's blast radius first.
2. Watch for the bill you're not being shown. The consumer's latency is on the dashboard. The producer's publish cost isn't — and it's the one that moved.

Neither is specific to the Disruptor. Any queue, executor or event loop with an idle consumer makes the same purchase — most just don't itemise it.

Sources. None of this is new: busy-spin versus block and the burned-core cost are standard mechanical sympathy, from Martin Thompson and the LMAX team through Nitsan Wakart and Aleksey Shipilëv on idle strategies. What's done here is reproducing the trade-off and putting numbers on both sides of it.

Code traces to LMAX Disruptor 3.4.4 at commit a87bf42: the busy-spin loop and its no-op signal at BusySpinWaitStrategy.java:36-48; the blocking waitFor at BlockingWaitStrategy.java:38-62 and its real signalAll at :66-77; the unconditional publish-time signal at SingleProducerSequencer.java:206-207; the single waitStrategy handed to every barrier by AbstractSequencer.newBarrier() at :108-111; the public SequenceBarrier interface at :23 and the constructor taking any implementation at BatchEventProcessor.java:54-57; and LiteBlockingWaitStrategy's elision at :78-92.

Every figure in the tables above ran on the same dedicated Xeon E5-2696 v4 (OpenJDK 21, performance governor, taskset), after the interference below was diagnosed and parked, with raw output and provenance in benchmarks/results/; the excursions quoted below come from the superseded runs in results/archive/. The hand-off harness is JMH-driven but sets no @BenchmarkMode, so its percentiles are computed by the harness over each iteration's ~50k samples.

On the deep tail. p99.9 is not the worst case for anyone here, because the machine has stalls of its own. Earlier runs showed busy-spin, yielding and sleeping all taking ~500 µs hits, and blocking one of ~489 µs — nothing mechanically in common, which gave the cause away. A kernel housekeeping task: the network driver refreshes its statistics every two seconds, stalls whichever core it lands on for ~0.5 ms, and prefers a busy one — the core that is spinning. Held on a spare core for the runs above, the worst stall in any measured iteration was 109 µs. No strategy removes it — spinning just catches more — which is why the statistic of record is p99.9, not max.

Three further residuals. One: blocking's 20.9 µs is the clean-rig figure; three earlier runs on this box put it 22–32% higher, for reasons the record does not yet explain, and a re-run is owed. Two: this Xeon exposes no turbo disable, and threads were not individually pinned — taskset confines the JVM to the physical cores, but the OS may still migrate the producer and consumer within them. Three: the no-consumer run that isolates the lock removes the wakeup and any lock contention together — so what it cleanly separates is an uncontended lock from everything a live consumer adds, not lock from wakeup.