Evicting Idle SSE Connections Safely

Part of Connection Pooling for SSE Servers, under Backend Stream Generation & Connection Management.

A registry full of connections nobody is reading is capacity spent on nothing — but “idle” and “abandoned” look identical from the server, and evicting the wrong one drops a user who was watching.

Symptom & Developer Intent

  • Open-stream count is far higher than plausible concurrent users.
  • The count never falls overnight, when real usage clearly does.
  • Descriptor usage grows until the process is restarted, and restarting “fixes” it.
  • Registry size and real socket count diverge over time.
  • Evicting on a simple age limit disconnects users who were actively reading.

The intent is a policy that reclaims dead connections quickly, leaves live ones alone, and tells anything it does evict how to reconnect.

Root Cause Analysis

Three different states produce a connection the server is holding needlessly.

Three states that all look like an open connection Table of half-open sockets, zombie registry entries and genuinely idle clients with how each looks to the server and what to do about it. Three states that all look like an open connection Looks like Correct action Half-open socket ESTABLISHED, no drain evict on drain failure Zombie entry registry > real FDs fix the cleanup path Idle but live drains every heartbeat leave it alone
Only one of the three is a policy question. The other two are bugs, and an evictor that papers over them will also evict people who were reading.

Half-open sockets. The client vanished without a FIN — a laptop lid closed, a NAT entry expired, a network dropped. The server’s socket remains in ESTABLISHED and writes succeed into the send buffer until it fills, which can take a long time on a quiet stream. Nothing indicates the peer is gone.

Zombie registry entries. The socket closed but the cleanup path did not run, so the entry survives its connection. This is the failure that makes registry size and descriptor count diverge, and it is a bug rather than a policy problem — the fix belongs in the disconnect handler, not in an evictor.

Genuinely idle but live clients. A user with a background tab open for six hours is a real client that is reading nothing. Evicting them frees capacity; evicting them badly makes the feed look broken when they return.

The distinguishing signal is write behaviour. A live client drains what you write; a half-open one accepts data into the network until buffers fill and then stops. That makes a heartbeat both a keep-alive and a liveness probe — its failure to drain is the evidence eviction needs.

Step-by-Step Resolution

Using the heartbeat as a liveness probe Progression of a connection through heartbeat writes: accepted writes mark it live, refused writes across several heartbeats mark it undrained and eligible for eviction. Using the heartbeat as a liveness probe Heartbeat sent every 15 s Accepted client is draining res.write() Refused ×6 buffer full, 90 s returns false Evicted end() + registry delete no drain a single accepted write returns the connection to healthy immediately
A write the socket accepts is proof somebody is reading. Age proves nothing — a six-hour-old stream can be perfectly healthy.

Step 1 — Fix the cleanup path before adding any evictor

An evictor that compensates for a missing cleanup handler hides the bug and evicts live connections as a side effect.

const registry = new Map();                 // id → { res, lastWriteOk, connectedAt }

app.get('/events', (req, res) => {
  const id = crypto.randomUUID();
  registry.set(id, { res, lastWriteOk: Date.now(), connectedAt: Date.now() });

  let released = false;
  const release = () => {
    if (released) return;
    released = true;
    registry.delete(id);
    unsubscribe(send);
  };

  req.on('close', release);                 // the normal path
  res.on('error', release);                 // EPIPE and friends
  res.on('finish', release);                // any deliberate end()
});

Step 2 — Use the heartbeat as a liveness probe

res.write() returning false means the socket buffer is full — the client is not draining.

function heartbeatAll() {
  const now = Date.now();
  for (const [id, entry] of registry) {
    let accepted;
    try {
      accepted = entry.res.write(': ping\n\n');
    } catch {
      registry.delete(id);                  // already dead
      continue;
    }
    if (accepted) entry.lastWriteOk = now;  // the client drained: definitively alive
  }
}
setInterval(heartbeatAll, 15_000);

Step 3 — Evict on failure to drain, not on age 

const DRAIN_TIMEOUT_MS = 90_000;            // several heartbeats' worth of grace

function evictUndrained() {
  const cutoff = Date.now() - DRAIN_TIMEOUT_MS;
  for (const [id, entry] of registry) {
    if (entry.lastWriteOk >= cutoff) continue;      // draining fine: leave alone
    metrics.inc('sse_evicted_total', { reason: 'no_drain' });
    try { entry.res.end(); } catch { /* already gone */ }
    registry.delete(id);
  }
}
setInterval(evictUndrained, 30_000);

A client that has not accepted a byte across six consecutive heartbeats is not reading. That is a far safer signal than connection age, which says nothing about whether anyone is there.

Step 4 — Add an age cap only as a capacity backstop 

const MAX_AGE_MS = 4 * 60 * 60 * 1000;

function evictAncient() {
  const cutoff = Date.now() - MAX_AGE_MS;
  for (const [id, entry] of registry) {
    if (entry.connectedAt > cutoff) continue;
    // Tell the client this is deliberate and when to return.
    entry.res.write('retry: 5000\n');
    entry.res.write('event: rotate\ndata: {"reason":"max_age"}\n\n');
    entry.res.end();
    registry.delete(id);
    metrics.inc('sse_evicted_total', { reason: 'max_age' });
  }
}

Jitter the cap per connection so long-lived clients do not all rotate together — the same reasoning as for serverless handovers.

Step 5 — Shed load by priority when near capacity 

const SOFT_LIMIT = 45_000;

function shedIfOverloaded() {
  if (registry.size < SOFT_LIMIT) return;
  const candidates = [...registry.entries()]
    .filter(([, e]) => e.hidden)                    // client reported a hidden tab
    .sort((a, b) => a[1].lastWriteOk - b[1].lastWriteOk);

  for (const [id, entry] of candidates.slice(0, registry.size - SOFT_LIMIT)) {
    entry.res.write('retry: 30000\n');              // come back later, not immediately
    entry.res.write('event: shed\ndata: {"reason":"capacity"}\n\n');
    entry.res.end();
    registry.delete(id);
  }
}

Clients that report a hidden tab — via the Page Visibility handling in Using the Page Visibility API to Pause Event Streams — are the right first candidates, because nobody is looking at them.

Step 6 — Handle eviction gracefully on the client 

es.addEventListener('rotate', () => { expectedClose = true; });
es.addEventListener('shed',   () => { expectedClose = true; });
es.addEventListener('error', () => {
  if (expectedClose) { expectedClose = false; return; }
  showReconnecting();
});

Validation & Monitoring

Connections reclaimed by policy Bar chart of connections reclaimed per hour by drain-failure eviction, the age cap and capacity shedding on a node holding fifty thousand streams. Connections reclaimed by policy one node, 50 000 streams, per hour Drain failure 420 Age cap 30 Capacity shedding 0 connections reclaimed per hour
Drain-failure eviction should dominate. An age cap doing most of the work means it is disconnecting people who were reading.
# 1. A half-open client must be evicted within the drain timeout.
curl -N -s localhost:8080/events > /dev/null &
PID=$!; sleep 5
sudo iptables -A OUTPUT -p tcp --sport 8080 -j DROP     # silently blackhole the path
sleep 120
curl -s localhost:9090/metrics | grep sse_evicted_total  # expect a no_drain eviction
sudo iptables -D OUTPUT -p tcp --sport 8080 -j DROP; kill $PID

# 2. A live but quiet client must NOT be evicted.
timeout 180 curl -N -s localhost:8080/events | grep -c ping   # expect ~12, no early end

That second test is the one that matters: an evictor which cannot distinguish quiet from dead will pass the first test and fail users.

# Evictions by reason — no_drain should dominate; max_age should be rare.
sum(rate(sse_evicted_total[15m])) by (reason)

# Registry versus reality: divergence means the cleanup path is leaking.
sum(sse_streams_open) - sum(process_open_fds{job="sse"})

Production Checklist

Frequently Asked Questions

How do I tell an idle client from a disappeared one?

By whether it drains what you write. A heartbeat that res.write() accepts and the socket clears is proof someone is reading; one that fills the send buffer and stops being accepted means nobody is. Connection age tells you nothing — a six-hour-old stream can be perfectly healthy.

Should I evict on a maximum connection age?

Only as a capacity backstop, jittered, and with a retry: hint so the client returns cleanly. Age alone is a poor signal of usefulness; drain failure is a good one. If an age cap is doing most of your eviction, it is probably disconnecting people who were reading.

My registry size does not match the descriptor count. Why?

The cleanup path is not running on every exit route — typically an error path that ends the response without firing the close handler. Make the release function idempotent and register it for close, error and finish. An evictor is not the fix; it just hides the leak.

Is it safe to shed load by disconnecting users?

Yes, if you tell them how to come back. Send a longer retry: value and an explanatory event so the client waits rather than reconnecting immediately, and prefer connections whose clients have reported a hidden tab. Shedding without a retry hint produces an immediate reconnect storm that makes the overload worse.