Fixing Silent Failures When Nginx Truncates Upstream Responses

Your API returns a perfectly valid JSON response from your backend server, but the client keeps getting a malformed or cut-off body. No 502, no 504, no error in your application logs — just a broken response that appears fine until someone actually reads it. This is one of the most frustrating classes of Nginx bugs because the failure is invisible by default.

The root cause is almost always Nginx's proxying and buffering layer quietly doing something you didn't intend. Once you know where to look, the fix takes minutes.

What you'll learn

• Why Nginx truncates upstream responses without logging an error
• How proxy buffering works and where it breaks down
• How to read the right Nginx debug signals
• The specific directives to tune for large or streaming responses
• How to verify your fix is actually working

Prerequisites

You should be comfortable editing Nginx configuration files and reloading the service. The examples here use a typical reverse-proxy setup where Nginx sits in front of a backend application (Node.js, Python, Java — it doesn't matter). You'll need access to Nginx error logs, ideally with debug or warn log level temporarily enabled.

Why Nginx Truncates Responses Silently

Nginx's proxy module buffers upstream responses in memory before forwarding them to the client. When a response fits inside the allocated buffers, everything is fine. When it doesn't, Nginx either spills to disk or closes the upstream connection — and depending on your configuration, it may close that connection before the client has received the full body.

The critical detail: this is not treated as an error by Nginx's default logging configuration. Nginx considers the job done once it starts forwarding. If it decides to stop buffering partway through, it moves on without writing anything to the error log at the default error level. That silence is what makes this so hard to catch.

There are three distinct scenarios that cause truncation:

• Buffer overflow with disk spill disabled or failing — the response is larger than the in-memory buffers, Nginx tries to write to a temp file, and the temp file path is not writable or disk space is exhausted.
• Upstream read timeout — the backend takes too long to send all the data, Nginx closes the upstream connection mid-stream.
• Upstream sends a premature close — less common, but your backend or a load balancer upstream closes the TCP connection before the final bytes arrive.

Reading the Logs You Actually Need

Before touching any config, get better visibility. Edit your Nginx log format to capture upstream response details. In your http or server block, add or modify your log format:

log_format detailed '$remote_addr - $remote_user [$time_local] '
                   '"$request" $status $body_bytes_sent '
                   '"$http_referer" "$http_user_agent" '
                   'ups_status=$upstream_status '
                   'ups_bytes=$upstream_response_length '
                   'ups_time=$upstream_response_time';

Apply this format to your access log with access_log /var/log/nginx/access.log detailed;. Now each request line shows you the upstream HTTP status, the number of bytes the upstream sent, and how long the upstream took to respond. Compare ups_bytes to the Content-Length header your application sends. If they differ, you've confirmed truncation.

For deeper diagnostics, temporarily set your error log level to warn or even debug:

error_log /var/log/nginx/error.log warn;

Reload with nginx -s reload and reproduce the problem. Look for lines containing upstream prematurely closed connection or no live upstreams while connecting to upstream. These give you a concrete starting point.

The Buffer Configuration Explained

Nginx's proxy buffering is controlled by a small set of directives. Understanding what each one does prevents you from tuning blindly.

The most common mistake is leaving all of these at defaults while your upstream starts sending responses that are larger than the combined buffer space. When the buffers are full and the temp file path isn't writable, Nginx quietly drops the connection.

Fixing Buffer-Related Truncation

For a typical API that returns responses up to a few megabytes, a reasonable starting configuration looks like this:

location /api/ {
    proxy_pass http://backend;

    proxy_buffering        on;
    proxy_buffer_size      16k;
    proxy_buffers          8 32k;
    proxy_busy_buffers_size 64k;

    proxy_temp_path        /var/cache/nginx/proxy_temp;
    proxy_max_temp_file_size 256m;
}

The values here are not universal — they depend on your typical response size. A good rule of thumb: set proxy_buffer_size large enough to hold the response headers plus the first chunk of body (headers are rarely more than 8–16k). Set proxy_buffers so the total buffer space (number × size) comfortably holds your median response. For a service that returns 200k JSON objects, eight 32k buffers gives you 256k total — enough headroom.

Make sure the proxy_temp_path directory exists and is owned by the Nginx worker user (usually www-data or nginx):

mkdir -p /var/cache/nginx/proxy_temp
chown nginx:nginx /var/cache/nginx/proxy_temp

If you're dealing with very large responses (file downloads, exports, reports), consider turning off buffering entirely for those locations. This tells Nginx to pass bytes from upstream to client as they arrive, which removes the buffer size problem entirely:

location /exports/ {
    proxy_pass    http://backend;
    proxy_buffering off;
}

The trade-off is that disabling buffering holds the upstream connection open until the client finishes downloading, which can exhaust upstream worker connections under heavy load. Use it selectively.

Fixing Timeout-Related Truncation

If your ups_time values in the access log are close to 60 seconds and responses are getting cut off, you're hitting the default proxy_read_timeout. This timeout governs how long Nginx waits between successive reads from the upstream — not the total response time. If your backend pauses between chunks of a large streamed response, a 60-second gap will close the connection.

location /api/ {
    proxy_pass http://backend;

    proxy_connect_timeout 10s;
    proxy_send_timeout    60s;
    proxy_read_timeout    120s;
}

Set proxy_read_timeout to match the realistic worst-case pause between upstream writes. For a backend that streams chunks every 30 seconds, 90–120 seconds gives you a sensible safety margin. Don't simply set it to a huge number (like 3600s) everywhere — you'll mask genuinely stalled connections and waste resources.

Diagnosing Upstream Premature Close

When the error log says upstream prematurely closed connection while reading response header from upstream, the backend is closing the TCP connection before sending a complete response. This is distinct from Nginx truncating things — the upstream is at fault here.

Common causes include:

• Gunicorn or uWSGI worker timeout killing a slow request mid-flight
• A Node.js uncaught exception closing the socket before the response is flushed
• A database query timeout causing the handler to exit early
• An intermediate load balancer (like an AWS ALB) with a shorter idle timeout than Nginx

To isolate this, bypass Nginx and hit the backend directly with curl -v http://backend-host:port/your-endpoint. If curl also gets a truncated or aborted response, the problem is in your application, not Nginx. Fix it there first.

Common Pitfalls

Changing buffers in the wrong context. Nginx directive inheritance means a setting in http {} applies everywhere, but a setting in a nested location {} overrides it. If you set generous buffers globally but then have a location block that resets proxy_buffering off, the location block wins. Always verify the effective configuration with nginx -T | grep proxy_buffer.

Forgetting to reload after editing. nginx -s reload performs a graceful reload. It does not interrupt in-flight requests, so if your test immediately after the edit still fails, it may be because existing worker processes are still handling that connection. Give it a few seconds or test with a fresh request.

Confusing proxy_read_timeout with request timeout. This timeout resets on every successful read from upstream. A backend that sends headers quickly and then slowly streams a large body will not hit this timeout as long as bytes keep arriving. If you're seeing truncation at exactly 60 seconds on slow streams, the backend is genuinely pausing for a full minute between writes — which is a backend bug, not an Nginx tuning problem.

Temp directory permissions after a system update. Package manager updates occasionally reset the permissions or ownership of Nginx cache directories. After an OS or Nginx package upgrade, verify the temp path is still writable.

Verifying the Fix

After making changes, verify with a targeted test. Use curl and compare the Content-Length header with the actual bytes received:

curl -so /dev/null -D - https://your-domain/api/large-endpoint \
  -w "bytes_downloaded: %{size_download}\nhttp_code: %{http_code}\n"

The size_download value should match the Content-Length in the response headers. If your backend sends chunked transfer encoding without a Content-Length, you'll need to compare against a known reference — either capture a direct backend response or check your application's own logs for how many bytes it wrote.

Also watch your custom access log for a few minutes of real traffic. The ups_bytes field should now be consistent and not suspiciously small on the requests that were previously truncated.

Wrapping Up

Silent truncation from Nginx usually comes down to one of three things: buffers that are too small, timeouts that are too short, or a backend that closes connections before it should. Here's what to do next:

• Add upstream_response_length and upstream_response_time to your Nginx access log format right now — even if you're not debugging a problem. This data is invaluable when something breaks at 2am.
• Audit your proxy_buffer_size and proxy_buffers against the actual response sizes your upstream returns. If you don't know what those sizes are, check your application metrics or add a Content-Length log to your backend.
• Test your backend directly with curl before blaming Nginx. If the issue reproduces without Nginx in the path, fix the application first.
• Review your proxy_read_timeout for any endpoints that stream data or run long queries. Match the timeout to the realistic worst-case gap between upstream writes, not the total response duration.
• After any Nginx or OS package update, verify that your proxy temp directory still exists and has correct ownership for the Nginx worker process.