Limiting SSE Connections per User

Part of Security Headers for Event Streams, under SSE Protocol Fundamentals & Architecture.

An unauthenticated rate limit protects the endpoint from strangers. A per-user connection cap protects it from your own customers — the tab-hoarder, the buggy client that leaks a stream per route change, and the script that opens a thousand connections with one valid token.

Symptom & Developer Intent

  • A handful of accounts hold a disproportionate share of open streams.
  • One buggy client version leaks connections and consumes a node’s descriptor budget.
  • Request-rate limiting has no effect, because each abusive client makes exactly one request.
  • After a node restart, users are locked out because stale counts were never released.
  • Rejections are indistinguishable from failures, so clients retry immediately and make it worse.

The intent is a cap that is accurate across nodes, releases reliably, and refuses politely.

Root Cause Analysis

Conventional rate limiting counts requests per interval. A stream is one request that lasts for hours, so a limiter measuring requests per minute sees a single event and permits it. The resource being consumed is concurrency, not request rate, and it needs a different mechanism: a counter that increments on connect, decrements on disconnect, and is shared across every node behind the load balancer.

Why a request-rate limiter sees nothing Table contrasting what a rate limiter and a concurrency limiter measure, and what an abusive streaming client looks like to each. Why a request-rate limiter sees nothing Measures Sees an abusive stream client as Rate limiter requests / interval one request — allowed Concurrency limiter open connections 1 000 open — refused
One request per stream is invisible to a per-minute limiter and can still be a thousand descriptors.

That introduces the two failure modes specific to concurrency limiting.

Lost decrements. A process that crashes, is OOM-killed or is force-terminated never runs its cleanup, so the count stays elevated forever and the user is locked out of a slot they are not using. Any distributed counter therefore needs an expiry, not just a decrement.

Check-then-act races. Reading a count and then incrementing it is not atomic; two simultaneous connections can both read four, both increment, and both proceed past a limit of five. The reserve must be a single atomic operation.

Step-by-Step Resolution

Reserving a slot atomically Sequence of the reserve operation: expire stale entries, add the connection, read the count, and release the slot immediately if the count exceeds the limit. Reserving a slot atomically Handler Redis ZREMRANGEBYSCORE (drop stale) ZADD conn (reserve) ZCARD → 6 ZREM conn (over limit)
Reserve first and give the slot back if it was over the limit — a read-then-write sequence lets two simultaneous connections both pass a limit of one.

Step 1 — Count concurrency, not requests, and expire it

A Redis sorted set per user gives atomic reservation, automatic expiry of stale entries, and an accurate count in one structure.

// Each connection is a member with a score of its last heartbeat time.
const LIMIT = 5;
const STALE_MS = 90_000;                       // longer than the heartbeat interval

async function reserveSlot(userId, connId) {
  const key = `sse:conns:${userId}`;
  const now = Date.now();

  const [, , count] = await redis
    .multi()
    .zremrangebyscore(key, 0, now - STALE_MS)  // drop entries from dead processes
    .zadd(key, now, connId)                    // reserve atomically
    .zcard(key)
    .expire(key, 300)                          // the key itself cannot leak
    .exec()
    .then((r) => r.map(([, v]) => v));

  if (count > LIMIT) {
    await redis.zrem(key, connId);             // over the cap: give the slot straight back
    return false;
  }
  return true;
}

async function releaseSlot(userId, connId) {
  await redis.zrem(`sse:conns:${userId}`, connId);
}

// The heartbeat refreshes the score, which is what keeps a live connection from expiring.
async function touchSlot(userId, connId) {
  await redis.zadd(`sse:conns:${userId}`, Date.now(), connId);
}

Expiry is what makes this crash-safe: a connection whose process died stops being refreshed and ages out of the set within STALE_MS.

Step 2 — Reject before the stream starts, with a real status code 

app.get('/events', async (req, res) => {
  const userId = req.user.id;
  const connId = crypto.randomUUID();

  if (!(await reserveSlot(userId, connId))) {
    // A real HTTP error, not a stream that opens and immediately closes.
    return res.status(429)
      .set('Retry-After', '30')
      .json({ error: 'too_many_streams', limit: LIMIT });
  }

  writeHeaders(res);
  const heartbeat = setInterval(() => {
    res.write(': ping\n\n');
    touchSlot(userId, connId).catch(() => {});
  }, 15_000);

  const release = () => {
    clearInterval(heartbeat);
    releaseSlot(userId, connId).catch(() => {});
  };
  req.on('close', release);
  res.on('error', release);
});

Returning 429 before the stream opens matters: EventSource treats a non-2xx as a permanent failure and stops retrying, which is exactly the behaviour you want for a client that is over its limit.

Step 3 — Make the client understand the rejection 

// EventSource cannot read the status code, so probe first when a limit exists.
async function openStream(url) {
  const probe = await fetch(url, { method: 'HEAD', credentials: 'include' });
  if (probe.status === 429) {
    showTooManyTabsMessage(Number(probe.headers.get('Retry-After') ?? 30));
    return null;
  }
  return new EventSource(url, { withCredentials: true });
}

Telling a user “you have this open in too many tabs” is far better than a silent failure that looks like an outage.

Step 4 — Reduce demand before enforcing the cap

A cap that users hit routinely is a design problem. Collapse per-feature streams into one multiplexed connection, and share across tabs where possible — see Diagnosing the Six-Connection Limit per Origin.

Step 5 — Set the limit from data, not intuition 

# What does the 99th-percentile legitimate user actually hold?
histogram_quantile(0.99, sum(rate(sse_conns_per_user_bucket[24h])) by (le))

Set the cap a few connections above that, so ordinary heavy users never notice it and a leak is caught quickly.

Validation & Monitoring

Twenty simultaneous connections against a limit of five Bar chart of accepted and refused connections when twenty requests arrive at once, under a check-then-act limiter and an atomic reserve. Twenty simultaneous connections against a limit of five 20 requests issued in parallel, limit of 5 Check-then-act: accepted 14 Check-then-act: refused 6 Atomic: accepted 5 Atomic: refused 15 connections
The race only appears under simultaneity, so a sequential test passes a limiter that is broken in exactly the case abuse produces.
# 1. The cap holds for sequential connections.
for i in $(seq 1 7); do
  curl -s -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $TOKEN" \
    -N https://api.example.com/events &
done
# Expect five 200s and two 429s.

# 2. The cap holds under a simultaneous burst (the race test).
seq 1 20 | xargs -P 20 -I{} curl -s -o /dev/null -w '%{http_code}\n' \
  -H "Authorization: Bearer $TOKEN" -N https://api.example.com/events | sort | uniq -c
# Expect exactly five 200s.

# 3. Slots are released when connections close.
redis-cli ZCARD "sse:conns:$USER_ID"    # should fall back to 0 after clients exit

# 4. Stale entries expire after a hard kill.
kill -9 $(pgrep -f sse-server); sleep 95
redis-cli ZCARD "sse:conns:$USER_ID"    # should be 0 without any cleanup running

Test four is the one that prevents a support queue: without expiry, a single crash locks every affected user out until someone flushes Redis by hand.

# Rejections should be rare and concentrated. A broad rise means the cap is too low
# or a client version is leaking connections.
sum(rate(sse_rejected_total{reason="too_many_streams"}[15m]))
count(count by (user_id) (sse_rejected_total{reason="too_many_streams"}))

Production Checklist

Frequently Asked Questions

Why does ordinary rate limiting not work for streams?

Because it measures requests per interval and a stream is one request. A client holding a thousand connections has made a thousand requests over a long period — well within any sane per-minute limit — while consuming a thousand descriptors. Concurrency is the resource, so concurrency is what has to be counted.

What happens to the count when a server crashes?

With a plain counter, nothing — the decrement never runs and users stay locked out. That is why the entries carry a timestamp and are expired: a connection whose process died stops being refreshed by heartbeats and ages out automatically within the stale window.

Should the limit be per user or per IP?

Per user for authenticated streams, because an office behind one NAT address shares an IP and would be limited collectively. Per IP is a reasonable additional layer for unauthenticated endpoints, but as the only control it punishes exactly the wrong people.

How does the client know it was rejected rather than disconnected?

It cannot read the status code from EventSource, so probe with a HEAD request before opening, or have the server accept the connection and immediately emit a rejected event explaining why. The probe is cleaner: a 429 keeps EventSource from retrying at all.