Troubleshooting 502 Errors on SSE Endpoints

Part of Proxy & CDN Configuration for SSE, under SSE Protocol Fundamentals & Architecture.

A 502 on a streaming endpoint almost never means the application is down. It means an intermediary gave up on a response it did not understand, and the specific reason is recoverable from the proxy’s error log in about a minute — if you know which line to look for.

Symptom & Developer Intent

  • EventSource fires onerror immediately, and the Network panel shows 502 Bad Gateway on the stream request.
  • The application’s own logs show the request arriving and the stream opening normally — sometimes even events being written.
  • curl directly against the application works perfectly; the same request through the proxy returns 502.
  • The 502 appears only for large events, or only after the stream has been open for a while, or only for some users.

The intent is to identify which hop produced the 502 and which of its limits was exceeded, rather than restarting the application in the hope that it helps.

Root Cause Analysis

A 502 is generated by a proxy when it cannot obtain a valid response from upstream. For streams there are five realistic causes, and they are distinguishable by when the 502 appears.

When the 502 appears tells you which cause it is Timeline placing the five causes of a 502 on a stream by the moment they occur relative to connection start. When the 502 appears tells you which cause it is time → Immediate headers too big / refused 0 ms On a large event buffer or temp file first event At the timeout proxy_read_timeout 60 s Mid-stream upstream restarted random
Timing is the diagnosis: immediate means headers or connectivity, a round number means a timeout, and payload-dependent means buffer sizing.

Immediately, before any event. The upstream connection failed or the response headers were rejected. Common triggers: the application is not listening, TLS verification failed on an internal hop, or the response header block exceeds the proxy’s buffer — an oversized Set-Cookie or a long CORS header set will do it at nginx’s default 4 KB proxy_buffer_size.

After exactly the upstream timeout. proxy_read_timeout elapsed with no bytes received. With buffering on, the proxy is waiting for a complete response that will never come; with buffering off but no heartbeat, an idle stream trips the same timer. The tell is a 502 at a round number of seconds — 60 by default in nginx.

On the first large event. With proxy_buffering on, an event larger than proxy_buffer_size spills to a temporary file, and if proxy_max_temp_file_size is zero or exceeded, nginx aborts with 502. This is why the failure can be payload-dependent, which sends people looking for a serialization bug that is not there.

Mid-stream, seemingly at random. The upstream closed the connection — a worker recycle, a deploy, an OOM kill — and the proxy reports 502 because it had nothing to forward. Correlate against application restarts before investigating the proxy at all.

Only over HTTP/2. An illegal hop-by-hop header from the application causes a protocol error that some proxies surface as 502. See HTTP/2 and HTTP/3 for Event Streams.

Step-by-Step Resolution

Reading the nginx error log Table mapping nginx upstream error messages to the cause and the configuration change that resolves it. Reading the nginx error log Means Fix prematurely closed upstream died check app restarts upstream timed out no bytes in window timeout + heartbeat too big header header block > buffer proxy_buffer_size 16k connection refused not listening check the upstream
The error log names the cause directly, which makes it the first thing to read and the last thing most people check.

Step 1 — Read the proxy error log, which names the cause directly 

sudo tail -f /var/log/nginx/error.log | grep -i 'upstream'

The message maps to the cause:

Log fragment Cause
upstream prematurely closed connection The application closed or crashed mid-stream
upstream timed out (110: Connection timed out) proxy_read_timeout elapsed with no bytes
upstream sent too big header Response headers exceed proxy_buffer_size
connect() failed (111: Connection refused) The application is not listening on that address
no live upstreams Health checks marked every backend down

Step 2 — Prove the application is innocent 

# From the proxy host itself, bypassing the proxy entirely.
curl -N -s -o /dev/null -w 'status:%{http_code} first_byte:%{time_starttransfer}s\n' \
  http://127.0.0.1:8080/events

A 200 with a fast first byte here and a 502 through the proxy narrows the problem to the proxy configuration with certainty.

Step 3 — Disable buffering, which removes three of the five causes at once 

location /events {
    proxy_pass http://sse_backend;

    proxy_buffering off;             # no accumulation, no temp files, no size limits
    proxy_request_buffering off;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
}

With buffering off, nginx forwards bytes as they arrive: proxy_buffers, proxy_busy_buffers_size and proxy_max_temp_file_size stop applying, so the payload-size and temp-file classes of 502 disappear.

Step 4 — Raise the header buffer if the log said “too big header” 

location /events {
    proxy_buffer_size   16k;         # header block only, when buffering is off
    proxy_buffers       8 16k;
    proxy_busy_buffers_size 32k;
}

Note that proxy_buffer_size still governs the response header block even when proxy_buffering is off — which is why a 502 with upstream sent too big header survives step 3.

Step 5 — Raise the read timeout above the heartbeat interval 

location /events {
    proxy_read_timeout 3600s;
    proxy_send_timeout 3600s;
}

And confirm the application actually sends heartbeats, because the timeout is only half the fix:

// Heartbeat must be comfortably inside the shortest timeout in the path.
const heartbeat = setInterval(() => res.write(': ping\n\n'), 15_000);
req.on('close', () => clearInterval(heartbeat));

Step 6 — Reload and confirm 

nginx -t && systemctl reload nginx

curl -N -s -o /dev/null -w 'status:%{http_code} first_byte:%{time_starttransfer}s\n' \
  https://api.example.com/events
# status:200 first_byte:0.08s

Validation & Monitoring

Confirm the fix survives the three conditions that produced the original failure: a large event, a long idle period, and sustained load.

Three conditions the fix must survive Verification pipeline covering a large payload, a long idle period and sustained concurrency. Three conditions the fix must survive a fix that passes only one of these has addressed only one cause Large event 64 KB payload Idle 3 min heartbeats only 200 streams all 200 OK buffer path timeout path
Each condition maps to one of the original causes, so passing all three is what distinguishes a fix from a coincidence.
# 1. Large event — publish something near your largest realistic payload.
curl -sX POST localhost:8080/admin/publish -d '{"size_kb": 64}'
curl -N -s https://api.example.com/events | head -c 200

# 2. Idle survival — no events for three minutes, only heartbeats.
timeout 200 curl -N -s https://api.example.com/events | grep -c ping
# Expect roughly 200/15 ≈ 13 heartbeats and no error.

# 3. Concurrency — many streams at once, watching for 502s.
for i in $(seq 1 200); do
  curl -N -s -o /dev/null -w '%{http_code}\n' https://api.example.com/events &
done | sort | uniq -c
# Expect: 200 × "200", no 502s.

For ongoing monitoring, alert on the proxy’s own status-code metric for the stream path rather than on application errors — the application never sees these failures.

# 502s on the stream path, from the proxy's perspective.
sum(rate(nginx_http_requests_total{status="502", location="/events"}[5m]))

A useful companion signal is p50 stream duration: if it collapses to a round number of seconds, a timeout somewhere is cutting streams even when it is not producing a visible 502.

Production Checklist

Frequently Asked Questions

Why does the 502 appear only for some events and not others?

Payload size. With proxy_buffering on, an event larger than proxy_buffer_size spills to a temporary file, and if temp files are disabled or the limit is exceeded nginx aborts with 502. Disabling buffering removes the size dependency entirely, which is why the fix looks unrelated to the symptom.

The 502 happens exactly 60 seconds after connecting. What is that?

The default proxy_read_timeout. The proxy received no bytes from upstream for 60 seconds and gave up. Two things fix it together: raise the timeout well above your heartbeat interval, and make sure the application actually sends heartbeat comments — raising the timeout alone just moves the failure later.

Can a 502 come from the CDN rather than from my proxy?

Yes, and the CDN's own origin timeout is usually shorter than anything you configured. Check whether the 502 body or headers carry the CDN's branding, and test the same URL against the origin directly. If the origin is clean and the edge returns 502, the edge's origin timeout or streaming support is the problem.

Should I just raise every buffer size instead of disabling buffering?

No. Larger buffers delay the failure rather than removing it, and they make the stream less real-time in the meantime, because the proxy still waits for a buffer to fill before forwarding. Disabling buffering is the correct configuration for a response that never ends; the buffer sizes then matter only for the header block.