Running FastAPI SSE Behind Gunicorn with Multiple Workers

Part of Python & FastAPI SSE Implementation Guide, under Backend Stream Generation & Connection Management.

A FastAPI stream that works perfectly with one worker starts delivering events to roughly one client in four the moment you scale to four workers. Nothing errored; the subscribers simply live in a different process from the publisher.

Symptom & Developer Intent

  • Events reach some connected clients and not others, apparently at random.
  • The proportion that receive an event matches one over the worker count.
  • A single-worker deployment is correct, and adding workers breaks it.
  • Restarting Gunicorn drops every stream at once.
  • Worker timeouts kill long-lived streams during quiet periods.

The intent is a multi-worker deployment where every subscriber receives every event, and where restarts drain rather than cut.

Root Cause Analysis

Gunicorn with Uvicorn workers forks separate OS processes, each with its own memory, its own event loop and its own module-level state. A set() of subscriber queues declared at module scope exists once per worker, so a publish handled by worker one reaches only the clients worker one is serving. With four workers each holding a quarter of the connections, three quarters of clients miss every event — which looks random because the load balancer distributed connections arbitrarily.

One publish, four separate process memories Gunicorn master with four Uvicorn workers, each holding its own in-memory subscriber set, so a publish handled by one worker reaches only that worker's clients. One publish, four separate process memories Publish handled by worker 2 Worker 1 2 500 clients — miss it Worker 2 2 500 clients — receive it Worker 3 2 500 clients — miss it Worker 4 2 500 clients — miss it
The fraction of clients that receive an event is one over the worker count — which is why the loss looks random rather than systematic.

Two Gunicorn defaults compound the problem.

The sync worker class cannot stream at all. Gunicorn’s default sync worker is not ASGI-aware; FastAPI requires uvicorn.workers.UvicornWorker, and using anything else produces buffering or outright failure.

The worker timeout kills quiet streams. Gunicorn’s --timeout (30 seconds by default) is a watchdog on worker responsiveness. With async workers it is less aggressive than with sync ones, but a worker whose event loop is blocked — by a synchronous database call inside a streaming generator, for example — is killed along with every stream it holds.

Restarts are the third issue: TERM to the master reaps workers after the graceful timeout, and streams are cut with no retry: hint, so every client reconnects at once.

Step-by-Step Resolution

Moving fan-out to a shared bus Corrected path where any worker publishes to Redis and every worker's subscriber connection receives it and writes to its own clients. Moving fan-out to a shared bus sse-starlette ping= supplies the heartbeat that keeps intermediaries happy Any worker PUBLISH Redis one delivery per worker Each worker own sub connection Its clients all of them from anywhere fan-out local write
Each worker needs its own subscribe connection, because a Redis connection in subscribe mode refuses every other command.

Step 1 — Use the Uvicorn worker class 

gunicorn app:app \
  --worker-class uvicorn.workers.UvicornWorker \
  --workers 4 \
  --bind 0.0.0.0:8000 \
  --timeout 0 \                  # no watchdog kill for long-lived requests
  --graceful-timeout 60 \        # time to drain streams on restart
  --keep-alive 65

--timeout 0 disables the watchdog entirely, which is appropriate when every request is long-lived by design; keep an eye on event-loop blocking through metrics instead.

Step 2 — Move subscribers out of process memory

Redis pub/sub is the smallest change that makes fan-out correct across workers.

# stream.py
import asyncio, json
import redis.asyncio as redis
from fastapi import APIRouter, Request
from sse_starlette.sse import EventSourceResponse

router = APIRouter()
publisher = redis.from_url("redis://localhost")     # for PUBLISH and ordinary commands

@router.get("/events")
async def events(request: Request, topic: str = "default"):
    # A SEPARATE connection: a connection in subscribe mode refuses other commands.
    subscriber = redis.from_url("redis://localhost")
    pubsub = subscriber.pubsub()
    await pubsub.subscribe(f"events:{topic}")

    async def generator():
        try:
            async for message in pubsub.listen():
                if await request.is_disconnected():
                    break
                if message["type"] != "message":
                    continue
                event = json.loads(message["data"])
                yield {"id": event["id"], "event": event["type"],
                       "data": json.dumps(event["data"])}
        finally:
            await pubsub.unsubscribe()
            await pubsub.close()
            await subscriber.close()

    return EventSourceResponse(generator(), ping=15)   # ping keeps intermediaries happy


async def publish(topic: str, event: dict) -> None:
    # Any worker can publish; every worker's subscribers receive it.
    await publisher.publish(f"events:{topic}", json.dumps(event))

The ping=15 argument is the heartbeat, and it is the reason sse-starlette is worth the dependency here.

Step 3 — Never block the event loop inside the generator

One synchronous call serialises every stream that worker holds, and a blocked loop is what the watchdog would have killed.

from fastapi.concurrency import run_in_threadpool

# WRONG: a sync driver call blocks the loop and every stream on this worker.
# row = db.execute("SELECT …").fetchone()

# RIGHT: push blocking work to a thread.
row = await run_in_threadpool(db.execute, "SELECT …")

# BETTER: use an async driver end to end.
row = await async_db.fetch_one("SELECT …")

Step 4 — Size workers against connections, not CPU

The usual 2 × cores + 1 heuristic assumes short requests. For streams, what matters is descriptors and memory per worker.

# Capacity per worker, roughly:
#   connections_per_worker = min(nofile_per_process - overhead, memory_budget / per_conn)
# 25 000 streams across 4 workers ≈ 6 250 per worker; each worker needs
# its descriptor limit raised accordingly (see the FD tuning guide).

# systemd unit — the limit must be raised on the process that actually serves.
[Service]
LimitNOFILE=131072
ExecStart=/usr/bin/gunicorn app:app --worker-class uvicorn.workers.UvicornWorker --workers 4

Step 5 — Drain streams on restart instead of cutting them 

import signal, asyncio

shutting_down = asyncio.Event()

def _handle_term(*_):
    shutting_down.set()

signal.signal(signal.SIGTERM, _handle_term)

async def generator():
    # Watch for shutdown alongside the message stream.
    while True:
        if shutting_down.is_set():
            yield {"event": "shutdown", "data": '{"reason":"deploy"}'}
            yield {"event": "retry", "retry": 15000}    # spread the reconnects
            break
        # … normal delivery …

Pair it with --graceful-timeout 60 so Gunicorn allows the drain to finish before reaping.

Validation & Monitoring

The decisive test needs more clients than workers.

Share of clients receiving each event Bar chart of the proportion of connected clients that receive a published event with in-memory fan-out at one, two and four workers, and with a shared bus. Share of clients receiving each event eight clients spread evenly across workers In-memory, 1 worker 100% In-memory, 2 workers 50% In-memory, 4 workers 25% Shared bus, any count 100% clients receiving each event
The single-worker case is the trap: it is completely correct, and it stops being correct the moment the deployment scales.
# 1. Connect eight clients across four workers, publish once, count receipts.
for i in $(seq 1 8); do curl -N -s localhost:8000/events?topic=t > /tmp/c$i.txt & done
sleep 1
curl -sX POST localhost:8000/publish -d '{"topic":"t","type":"tick","data":{"n":1}}'
sleep 1
grep -l '"n": 1' /tmp/c*.txt | wc -l
# Must be 8. Anything less — typically 2 with four workers — means in-memory fan-out.

# 2. Quiet streams survive well past the old worker timeout.
timeout 120 curl -N -s localhost:8000/events?topic=t | grep -c ping   # expect ~8

# 3. A restart drains rather than cutting.
sudo systemctl reload gunicorn
grep -c 'event: shutdown' /tmp/c1.txt    # expect 1

# Per-worker balance: a large skew means the load balancer is not spreading connections.
sse_streams_open

# Event-loop lag per worker — the signal that something is blocking the loop.
histogram_quantile(0.99, sum(rate(asyncio_event_loop_lag_seconds_bucket[5m])) by (le, worker))

Production Checklist

Frequently Asked Questions

Why do only some clients receive events after adding workers?

Because each Gunicorn worker is a separate process with its own memory. A module-level set of subscriber queues exists once per worker, so a publish handled by one worker reaches only the clients that worker is serving. The fraction that receive it is roughly one over the worker count — move fan-out to Redis or another shared bus.

Can I avoid Redis by running a single worker?

You can, and it is a legitimate choice for small deployments: one worker, one event loop, in-memory fan-out, scaled by running more single-worker instances behind a load balancer. But those instances have the same problem between them, so the moment you run two you need the shared bus anyway.

What should the Gunicorn timeout be for streams?

Zero, or far above any session length. The timeout is a watchdog for unresponsive workers, and with async workers a long-lived request is not unresponsiveness. Disable it and monitor event-loop lag instead — that is the signal that actually detects a blocked worker.

Why does the subscriber need its own Redis connection?

Because a connection in subscribe mode refuses ordinary commands. Sharing one connection between the pub/sub listener and the rest of the application produces errors on unrelated calls, so the listener gets its own connection and everything else uses the shared pool.