Handling Execution Timeouts in Serverless SSE

Part of Edge & Serverless SSE Deployment, under Backend Stream Generation & Connection Management.

Every serverless platform ends a long invocation eventually. The difference between a stream that rolls over invisibly and one that produces a synchronised reconnect storm every fifteen minutes is about twenty lines of code.

Symptom & Developer Intent

  • Streams end at the same interval for every user, matching the platform’s execution limit exactly.
  • The reconnect wave arrives all at once, because clients that connected together time out together.
  • Users see a “reconnecting” indicator on a fixed schedule, which erodes trust in the indicator.
  • Events published during the reconnect gap are missing from the client’s view.
  • Metrics cannot distinguish a planned rollover from a real failure, so disconnect rates look alarming.

The intent is a handover the user never notices: closed deliberately, spread over time, resumed without a gap.

Root Cause Analysis

An execution ceiling is a property of the platform, not a fault. Three things turn it into a visible problem.

Reconnects arriving in the first ten seconds after expiry Bar chart of peak reconnects per second when streams expire together versus with per-connection jitter applied to the handover time. Reconnects arriving in the first ten seconds after expiry 20 000 streams that connected within the same minute No jitter 20 000 ±10% jitter 3 300 ±20% jitter 1 700 peak reconnects per second
Clients that connected together expire together. A ±20% spread costs nothing and turns the wave into a ramp.

Synchronised expiry. Clients that connect at the same moment — after a deploy, or when a popular page loads — reach the ceiling at the same moment. Without jitter, every one of them reconnects simultaneously, producing a load spike on a schedule.

No advance warning to the client. When the runtime kills the invocation, the connection simply closes. EventSource treats that as an error and applies whatever retry interval it last saw — the browser default of a few seconds if the server never sent one. The UI shows a failure that was not one.

A replay window too small for the gap. The reconnect gap is not just the retry interval: it includes DNS, TLS, a possible cold start, and re-subscription. If the bus retains fewer events than are published during that window, every handover loses data — and the loss is invisible unless the client detects the id discontinuity.

Step-by-Step Resolution

One handover cycle, from the client's side State cycle of a client through a planned handover: live, handover announced, brief reconnect, replay, live again. One handover cycle, from the client's side Live events flowing Handover server announces deadline reached Reconnect retry + cold start Last-Event-ID Replayed gap filled, live buffer covers gap the cycle repeats every ten to fifteen minutes, invisibly
Because the client is told the close is deliberate, the connection UI stays quiet and the replay closes the gap — the user sees nothing at all.

Step 1 — Choose a handover point with real margin

Close well before the ceiling so a slow final write cannot be truncated.

// Handover at ~85% of the ceiling, with an absolute floor of one minute of margin.
const CEILING_MS = Number(process.env.PLATFORM_CEILING_MS ?? 900_000);
const MARGIN_MS = Math.max(60_000, CEILING_MS * 0.15);
const HANDOVER_MS = CEILING_MS - MARGIN_MS;

Step 2 — Jitter the handover per connection

This is the single most important line: it turns a synchronised wave into a smooth trickle.

// ±20% jitter so simultaneous connections do not expire simultaneously.
const jitter = 1 + (Math.random() * 0.4 - 0.2);
const handoverAt = Date.now() + HANDOVER_MS * jitter;

Step 3 — Send a retry hint at stream start, not at close

The client must already know the interval before the connection ends, because it cannot read anything afterwards.

// First bytes on the wire: control the reconnect delay up front.
stream.write('retry: 2000\n\n');

Step 4 — Announce the handover, then close cleanly 

for await (const ev of subscription) {
  stream.write(`id: ${ev.id}\nevent: ${ev.type}\ndata: ${JSON.stringify(ev.data)}\n\n`);

  if (Date.now() > handoverAt) {
    stream.write(
      `event: handover\ndata: ${JSON.stringify({
        reason: 'max_duration',
        resume_from: ev.id,          // the client already has this; be explicit anyway
      })}\n\n`,
    );
    break;
  }
}
stream.end();

A stream with no events would never reach the check, so run the deadline on the heartbeat too:

const heartbeat = setInterval(() => {
  if (Date.now() > handoverAt) {
    stream.write('event: handover\ndata: {"reason":"max_duration"}\n\n');
    clearInterval(heartbeat);
    stream.end();
    return;
  }
  stream.write(': ping\n\n');
}, 15_000);

Step 5 — Keep the UI quiet for a planned close 

// Client: a handover is not a failure and must not surface as one.
let plannedClose = false;

es.addEventListener('handover', () => { plannedClose = true; });

es.addEventListener('error', () => {
  if (plannedClose) { plannedClose = false; return; }   // expected rollover
  showReconnectingBanner();                             // a real problem
});

es.addEventListener('resync', async () => {
  applySnapshot(await (await fetch('/api/snapshot')).json());
});

Step 6 — Size the replay window to the real gap 

// Retain at least: retry delay + cold start + subscribe time, times the event rate,
// with generous headroom.
const GAP_SECONDS = 2 /* retry */ + 3 /* cold start */ + 1 /* subscribe */;
const EVENTS_PER_SECOND = 50;
const MIN_REPLAY = GAP_SECONDS * EVENTS_PER_SECOND * 4;   // ≈1200 events

// If the requested cursor has aged out, say so rather than silently starting from now.
if (!buffer.has(cursor)) {
  stream.write('event: resync\ndata: {"reason":"buffer_expired"}\n\n');
}

Validation & Monitoring

Events to replay across a reconnect gap Bar chart of the number of events published during a reconnect gap at fifty events per second, broken down by the components of the gap. Events to replay across a reconnect gap 50 events/s published during the gap retry 2 s 100 + cold start 3 s 250 + subscribe 1 s 300 recommended buffer 1 200 events the buffer must retain
The gap is not the retry interval alone. Sizing the buffer for the retry value only is the reason resyncs appear at every handover.
# 1. Do streams end at the handover point rather than the ceiling?
for i in 1 2 3; do
  start=$(date +%s)
  curl -N -s "$STREAM_URL" > /dev/null
  echo "stream $i lasted $(( $(date +%s) - start ))s"
done
# Expect values scattered around the handover point — the jitter should be visible.

# 2. Does the handover event arrive before the close?
curl -N -s "$STREAM_URL" | grep -m1 'event: handover'

# 3. Is the reconnect gap actually covered by the replay window?
curl -N -s -H 'Last-Event-ID: 4821' "$STREAM_URL" | head -3
# Expect 4822 next, with no gap and no resync event.

Separate planned from unplanned disconnects in metrics, or the disconnect rate is uninterpretable:

streamDuration.observe({ reason: 'max_duration' }, seconds);   // planned handover
streamDuration.observe({ reason: 'client_closed' }, seconds);  // the user left
streamDuration.observe({ reason: 'write_error' }, seconds);    // a real failure
# Share of disconnects that were planned — should be the large majority on serverless.
sum(rate(sse_stream_duration_seconds_count{reason="max_duration"}[15m]))
  / sum(rate(sse_stream_duration_seconds_count[15m]))

# Resyncs mean the replay window is too small for the real gap.
sum(rate(sse_resync_total[15m]))

A non-zero resync rate is the clearest actionable signal here: it means users are losing data at every handover, and the fix is a larger buffer rather than anything on the client.

Production Checklist

Frequently Asked Questions

Why jitter the handover if every stream has the same ceiling anyway?

Because expiry time is set by connect time, and connect times cluster — after a deploy, or when a popular page loads. Without jitter those clients all expire together and reconnect together, recreating the load spike on a schedule. A ±20% spread turns the wave into a trickle at no cost.

Can I keep the connection alive past the platform ceiling somehow?

No. The ceiling terminates the invocation regardless of what the handler does. The only options are to hand over deliberately before it, or to move continuously-open streams to a runtime without one — an isolate platform or a persistent server.

How do users perceive a handover every ten to fifteen minutes?

Not at all, provided three things hold: the retry delay is short, the replay window covers the gap, and the UI does not announce the reconnect. Users notice handovers when a reconnect banner flashes on a schedule or when data goes missing — both are symptoms of one of those three being wrong.

Should the client reconnect immediately or wait?

Wait briefly — one to two seconds via the retry: field. Immediate reconnection from every client at once is exactly the thundering herd the jitter is designed to avoid, and a short delay costs the user nothing because the replay makes the gap invisible.