Tuning AWS ALB Idle Timeouts for SSE

Part of Proxy & CDN Configuration for SSE, under SSE Protocol Fundamentals & Architecture.

A stream that dies at almost exactly 60 seconds behind an AWS load balancer is not a coincidence and not a bug in your application: it is the ALB’s default idle timeout doing precisely what it is documented to do.

Symptom & Developer Intent

  • Streams disconnect after 60 seconds, reconnect, and disconnect again 60 seconds later, indefinitely.
  • The pattern is exact enough to set a watch by, and it disappears when testing against the instance directly.
  • Application logs show the request closing with no error; the client sees a plain reconnect.
  • Reconnect counts in client telemetry equal session length divided by sixty.
  • During a deploy, every stream drops at once even though the new targets are healthy.

The intent is to keep streams alive for their natural session length behind the ALB, and to drain them deliberately during deploys rather than having them cut.

Root Cause Analysis

An ALB closes any connection with no data transfer in either direction for longer than its idle timeout, which defaults to 60 seconds. The timer is reset by bytes flowing in either direction — so it is not a maximum connection lifetime, only a maximum silence. A busy stream never trips it; an idle one always does.

The sixty-second cycle Timeline of a quiet stream behind an ALB with the default idle timeout: silence, disconnection at sixty seconds, reconnect, and the same again. The sixty-second cycle time → Stream opens no events yet 0 s ALB closes it idle timeout 60 s Client reconnects retry interval 63 s And again the cycle repeats 123 s
The interval is exact because it is a configured default, not a network condition — which is what makes it identifiable from the metric alone.

This produces the exact-60-second signature, and it explains why the problem is invisible in load tests: synthetic traffic keeps events flowing, so the timer never expires. Real users on a quiet feed hit it every minute.

Two related settings extend the same failure mode. Deregistration delay (default 300 seconds) determines how long a target keeps serving existing connections after being removed from rotation; if it is shorter than your drain window, streams are cut mid-deploy. And target group health checks pointed at a streaming path never receive a complete response, so the check times out and the target is marked unhealthy — removing a node that is working perfectly.

There is one more constraint worth knowing: the ALB idle timeout has a documented maximum of 4000 seconds. Sessions longer than that will always be cut, which makes an application-level heartbeat mandatory rather than optional.

Step-by-Step Resolution

Four settings, in the order they matter Stack of the application heartbeat, ALB idle timeout, deregistration delay and health check target, with the value each needs. Four settings, in the order they matter App heartbeat : ping every 15 s works at every hop, including yours ALB idle timeout default 60 s raise to 4000 s Deregistration delay default 300 s above the drain window Health check path not the stream a plain /healthz endpoint set it to
The heartbeat is first because it is the only one that also covers the NAT devices and firewalls whose timeouts you cannot configure.

Step 1 — Send heartbeats from the application

This is the fix that works everywhere, including the NAT devices and corporate firewalls whose timeouts you cannot configure. Everything else is defence in depth.

// A comment line resets every idle timer in the path without dispatching an event.
const HEARTBEAT_MS = 15_000;                 // comfortably inside a 60s timeout

app.get('/events', (req, res) => {
  res.writeHead(200, {
    'Content-Type': 'text/event-stream; charset=utf-8',
    'Cache-Control': 'no-cache, no-transform',
  });
  res.flushHeaders();

  const heartbeat = setInterval(() => res.write(': ping\n\n'), HEARTBEAT_MS);
  req.on('close', () => clearInterval(heartbeat));   // never outlive the socket
});

Fifteen seconds gives four heartbeats inside the default timeout — enough that a single dropped write does not cost the connection.

Step 2 — Raise the ALB idle timeout 

aws elbv2 modify-load-balancer-attributes \
  --load-balancer-arn "$ALB_ARN" \
  --attributes Key=idle_timeout.timeout_seconds,Value=4000

# Confirm it applied.
aws elbv2 describe-load-balancer-attributes --load-balancer-arn "$ALB_ARN" \
  --query "Attributes[?Key=='idle_timeout.timeout_seconds']"

In Terraform:

resource "aws_lb" "api" {
  name               = "api"
  load_balancer_type = "application"
  idle_timeout       = 4000     # seconds; 60 is the default that cuts streams
}

Step 3 — Extend the deregistration delay to cover the drain window 

aws elbv2 modify-target-group-attributes \
  --target-group-arn "$TG_ARN" \
  --attributes Key=deregistration_delay.timeout_seconds,Value=600

Set it above the time your application takes to drain streams deliberately, so the load balancer is never the thing that cuts them. The drain itself belongs in the application — see Graceful Shutdown for Go SSE Servers for the pattern.

Step 4 — Point health checks at a non-streaming endpoint 

aws elbv2 modify-target-group \
  --target-group-arn "$TG_ARN" \
  --health-check-path /healthz \
  --health-check-interval-seconds 15 \
  --health-check-timeout-seconds 5

// A health endpoint that reports stream-subsystem health without streaming.
app.get('/healthz', (_req, res) => {
  res.json({
    ok: bus.connected && streamsOpen < MAX_STREAMS,
    streams_open: streamsOpen,
    last_publish_ms_ago: Date.now() - lastPublishAt,
  });
});

Step 5 — Drain streams during deploys instead of letting them be cut 

// On SIGTERM: stop accepting, tell clients when to come back, then close.
process.on('SIGTERM', async () => {
  server.close();                                   // stop accepting new connections
  for (const client of registry) {
    client.write('retry: 15000\n');                 // spread reconnects over 15s
    client.write('event: shutdown\ndata: {"reason":"deploy"}\n\n');
    client.end();
  }
  await sleep(2_000);                               // let the writes flush
  process.exit(0);
});

Without the jittered retry hint, every disconnected client reconnects at the same instant against the replacement targets.

Validation & Monitoring

The decisive test is an idle stream, because a busy one proves nothing.

How long an idle stream survives Bar chart of idle stream lifetime under the default timeout, a raised timeout, and a raised timeout with heartbeats. How long an idle stream survives no events flowing, only heartbeats where enabled Default, no heartbeat 60 s Raised, no heartbeat 4 000 s Raised + heartbeat 14 400 s idle stream lifetime
Raising the timeout alone moves the failure later; the heartbeat is what removes it, because it resets every timer in the path including the ones you do not own.
# 1. Idle survival past the old 60-second cliff, with no events flowing.
start=$(date +%s)
timeout 300 curl -N -s https://api.example.com/events > /dev/null
echo "stream lasted $(( $(date +%s) - start ))s"
# Expect 300 (the timeout), not 60.

# 2. Heartbeats are actually arriving at the expected cadence.
timeout 65 curl -N -s https://api.example.com/events | grep -c '^: ping'
# Expect about 4 at a 15-second interval.

# 3. During a deploy, confirm clients receive the shutdown event rather than a bare close.
curl -N -s https://api.example.com/events | grep -m1 'event: shutdown'

For continuous signal, watch the p50 of stream duration. Before the change it sits at exactly 60 seconds; afterwards it should reflect real session lengths. A collapse back to a round number is the fastest way to notice that an infrastructure change reintroduced a short timeout somewhere.

# Median stream lifetime — a round number here is a timeout, not user behaviour.
histogram_quantile(0.5, sum(rate(sse_stream_duration_seconds_bucket[30m])) by (le))

Pair it with the ALB’s own TargetConnectionErrorCount and HTTPCode_ELB_5XX_Count metrics, which distinguish load-balancer-generated failures from application ones.

Production Checklist

Frequently Asked Questions

Is raising the idle timeout enough on its own?

No. The ALB caps at 4000 seconds, and the NAT devices and corporate firewalls between your users and AWS have their own timeouts that you cannot configure — commonly around five minutes with no notification. The heartbeat is the fix that works at every hop; raising the ALB timeout is defence in depth for the hop you control.

Why do all my streams drop simultaneously during a deploy?

Targets are deregistered and their connections are closed once the deregistration delay elapses. Raise that delay above your drain window and drain deliberately on SIGTERM, writing a jittered retry: value before closing so reconnects spread out instead of arriving as one wave against the new targets.

Can I use a Network Load Balancer instead to avoid this?

An NLB operates at layer 4 and has a fixed 350-second idle timeout that cannot be changed, so it does not remove the need for heartbeats — and it gives up the request routing, header handling and HTTP metrics an ALB provides. For SSE the ALB with a raised timeout plus heartbeats is the better arrangement.

What heartbeat interval should I choose?

Around a quarter of the shortest timeout in the path. With a 60-second floor somewhere, 15 seconds gives four chances before the timer expires. Below ten seconds the egress cost starts to be measurable at scale for no additional safety; above 30 seconds a single lost write can cost the connection.