What sets the tipping point?

Utilization ρ = arrival_rate / service_rate (λ/μ). Below 1 the server has spare capacity and the backlog always drains; at ρ = 1 there is exactly no slack; above 1 the leftover (λ − μ) work piles up every tick and the queue grows without bound. The default sits at ρ ≈ 1.1, the calm contrast at ρ ≈ 0.75.

Why does a queue at ρ just below 1 still build up during bursts?

The mean arrival rate decides whether the backlog is bounded, but the instantaneous rate swings above μ on each burst. During a spike work queues faster than the server can clear it; it only drains back to zero because the trough that follows runs below capacity. Higher mean utilization leaves less slack to recover in, which is why latency climbs steeply as ρ → 1.

What is the real-world analog?

This is the fluid version of an M/M/1 queue — one server, one waiting line. The same ρ → 1 blow-up governs a checkout lane, a CPU run-queue, a Kafka partition with one consumer, or a help desk: the closer you run to 100% utilization, the longer the line and the more a small burst hurts.

Cut ρ below 1 — add server capacity (raise μ), shed or smooth arrivals (lower λ or its burstiness), or add a parallel consumer. The calm config does exactly this by lowering producer_rate_base so μ once again exceeds λ.

Why does throughput flatten out under overload?

Throughput is capped at the service rate μ. Once the backlog exceeds what the server can clear in a tick, the server runs flat-out at μ forever, so throughput pins to 100 while the backlog keeps growing — useful work is maxed out, yet the line still explodes.

Queue Simulation

A single-server queue at the edge of overload: nudge utilization past 1 and the backlog diverges.

Level: Beginner

queue throughput bottleneck discrete-event service capacity

Stocks: backlog
Flows: arrivals, served
Probes: backlog, utilization, throughput

Run

Delays

Learn about delays in systems, how they create oscillations, and their impact on system behavior and decision-making.

Take the Quiz

simulation.py

A single queue at the edge of overload

Work arrives at one server that can handle a fixed number of jobs per second. The whole story is utilization, the ratio ρ = arrival_rate / service_rate. When ρ is below 1 the server keeps up and the backlog stays bounded; push the arrival rate past the service rate and ρ crosses 1, so the backlog grows without bound — there is no steady state to settle into.

The default sits just over the edge (ρ ≈ 1.1) and the calm contrast just under it (ρ ≈ 0.75). One knob, producer_rate_base, flips a queue that drains into one that diverges.


import math
from tys import probe, progress

Simulate a single-server queue fed by a bursty arrival stream.

def simulate(cfg: dict):

    import simpy
    env = simpy.Environment()

Parameters

    arrival_base = cfg["producer_rate_base"]   # mean arrivals / sec  (λ)
    arrival_amp  = cfg["producer_rate_amp"]    # burst amplitude
    arrival_freq = cfg["producer_rate_freq"]   # burst cycles / sec
    service_rate = cfg["consumer_rate"]        # max served / sec     (μ)
    sim_time     = cfg["sim_time"]
    backlog      = cfg.get("initial_backlog", 0)

    done = env.event()

The arrival rate is a sine burst riding on the mean λ, so even the calm queue sees momentary spikes — the mean is what decides its fate.

    def arrival_rate(time_sec: float) -> float:
        return max(
            0.0,
            arrival_base + arrival_amp * math.sin(2 * math.pi * arrival_freq * time_sec),
        )

Each tick: new work joins the queue, then the server runs flat-out, serving up to μ jobs. Below capacity it clears the backlog; above it, the leftover (λ − μ) piles up every single tick.

    def dynamics():
        nonlocal backlog
        warned = False
        for t in range(sim_time):
            arrivals = arrival_rate(t)
            backlog += arrivals
            served = min(backlog, service_rate)
            backlog -= served

            rho = arrivals / service_rate          # offered load ρ = λ/μ
            probe("backlog", env.now, backlog)
            probe("utilization", env.now, rho)
            probe("throughput", env.now, served)

            if rho > 1 and not warned:
                progress(int(100 * t / sim_time),
                         "ρ > 1 — arrivals outrun the server, backlog diverging")
                warned = True

            yield env.timeout(1)

        progress(100)
        mean_rho = arrival_base / service_rate
        done.succeed({
            "final_backlog":    backlog,
            "mean_utilization": mean_rho,

A bounded queue ends near where it started; a diverging one is many service-times deep by the end of the window.

            "diverged":         backlog > 5 * service_rate,
        })

    env.process(dynamics())
    env.run(until=done)
    return done.value


def requirements():
    return {
        "external": ["simpy==4.1.1"],
    }

Overload (ρ≈1.1).yaml

# overload.yaml — the default, just past the edge: ρ = λ/μ = 110/100 ≈ 1.1.
# Arrivals outrun the server, so the backlog grows without bound.
producer_rate_base: 110
producer_rate_amp: 40
producer_rate_freq: 0.01
consumer_rate: 100
sim_time: 300
initial_backlog: 0

Charts (Overload (ρ≈1.1))

backlog

Samples	300 @ 0.00–299.00
Values	min 10.00, mean 2141.41, median 2110.10, max 3797.85, σ 976.06

utilization

Samples	300 @ 0.00–299.00
Values	min 0.70, mean 1.10, median 1.10, max 1.50, σ 0.28

throughput

Samples	300 @ 0.00–299.00
Values	min 100.00, mean 100.00, median 100.00, max 100.00, σ 0.00

Final Results (Overload (ρ≈1.1))

Metric	Value
final_backlog	3000.00
mean_utilization	1.10
diverged	true

Calm (ρ≈0.75).yaml

# calm.yaml — the contrast: only producer_rate_base changes (110 -> 75),
# so ρ = 75/100 = 0.75. Bursts queue briefly but the server always catches up.
producer_rate_base: 75
producer_rate_amp: 40
producer_rate_freq: 0.01
consumer_rate: 100
sim_time: 300
initial_backlog: 0

Charts (Calm (ρ≈0.75))

backlog

Samples	300 @ 0.00–299.00
Values	min 0.00, mean 69.79, median 0.00, max 281.22, σ 101.29

utilization

Samples	300 @ 0.00–299.00
Values	min 0.35, mean 0.75, median 0.75, max 1.15, σ 0.28

throughput

Samples	300 @ 0.00–299.00
Values	min 35.00, mean 75.00, median 86.15, max 100.00, σ 26.34

Final Results (Calm (ρ≈0.75))

Metric	Value
final_backlog	0.00
mean_utilization	0.75
diverged	false

FAQ

What sets the tipping point?: Utilization ρ = arrival_rate / service_rate (λ/μ). Below 1 the server has spare capacity and the backlog always drains; at ρ = 1 there is exactly no slack; above 1 the leftover (λ − μ) work piles up every tick and the queue grows without bound. The default sits at ρ ≈ 1.1, the calm contrast at ρ ≈ 0.75.
Why does a queue at ρ just below 1 still build up during bursts?: The mean arrival rate decides whether the backlog is bounded, but the instantaneous rate swings above μ on each burst. During a spike work queues faster than the server can clear it; it only drains back to zero because the trough that follows runs below capacity. Higher mean utilization leaves less slack to recover in, which is why latency climbs steeply as ρ → 1.
What is the real-world analog?: This is the fluid version of an M/M/1 queue — one server, one waiting line. The same ρ → 1 blow-up governs a checkout lane, a CPU run-queue, a Kafka partition with one consumer, or a help desk: the closer you run to 100% utilization, the longer the line and the more a small burst hurts.
What stabilizes it?: Cut ρ below 1 — add server capacity (raise μ), shed or smooth arrivals (lower λ or its burstiness), or add a parallel consumer. The calm config does exactly this by lowering producer_rate_base so μ once again exceeds λ.
Why does throughput flatten out under overload?: Throughput is capped at the service rate μ. Once the backlog exceeds what the server can clear in a tick, the server runs flat-out at μ forever, so throughput pins to 100 while the backlog keeps growing — useful work is maxed out, yet the line still explodes.

Nested Hierarchy Workflow Ride-Share Surge Control

Open in Playground