Getting ChatGPT to Write Accurate API Rate Limit Headers Without Spec Gaps

You ask ChatGPT to add rate limiting to your API. It produces a middleware that sets X-RateLimit-Limit and X-RateLimit-Remaining — and then stops. No reset timestamp. No Retry-After. No client-side example showing how to actually read and respect those headers. Your API ships, clients ignore the headers because they don't know what to do with them, and the first traffic spike hits you like it was never there.

The problem isn't that ChatGPT can't write rate limit code. It can. The problem is that without a precise prompt, it generates the happy path and treats the rest as implied. Rate limiting is one of those features where the implied parts are exactly where real-world failures live.

What You'll Learn

• Which headers belong in a complete rate limit response and why ChatGPT drops them
• How to write a prompt that locks in the full header spec upfront
• How to handle Retry-After correctly for both HTTP-date and delta-seconds formats
• How to get ChatGPT to produce client-side parsing logic alongside the server code
• Common spec gaps to verify before merging AI-generated rate limit code

Prerequisites

You should be comfortable reading HTTP response headers and have a basic familiarity with middleware patterns. The examples use Python (FastAPI) and JavaScript (Node/Express), but the prompting technique applies to any stack. You'll need a ChatGPT account; GPT-4-class models produce noticeably better results than GPT-3.5 for this kind of structured output.

Why ChatGPT Stumbles on Rate Limit Headers

ChatGPT's training data contains a lot of rate limit middleware examples, but most of them are tutorials that stop at the middleware layer. They show the server counting requests and returning a 429 status — full stop. The client-side contract, the header semantics, and the edge cases around window resets rarely appear in the same article.

So when you ask for "rate limiting middleware," ChatGPT pattern-matches to those tutorial examples. It gives you a working counter and a 429, but it omits anything that wasn't consistently present in its training examples: the X-RateLimit-Reset epoch, the Retry-After header on 429 responses, the distinction between fixed windows and sliding windows, and any guidance for the API client.

The fix is to describe the full contract in your prompt, not just the feature name. ChatGPT is good at implementing specs; it just needs you to provide one.

The Header Set ChatGPT Usually Forgets to Complete

A production-grade rate limit implementation should send this set of headers on every response — not just on 429s:

ChatGPT reliably generates the first two. It sometimes generates the third. It rarely generates X-RateLimit-Window, and it almost never generates a correctly formatted Retry-After on the 429 response unless you explicitly ask for it.

The Retry-After header is particularly important because it's the one clients can act on programmatically. Without it, a client receiving a 429 has to guess when to retry, which usually means either hammering your server or implementing its own exponential backoff from scratch. For more on how ChatGPT handles backoff, see getting ChatGPT to write accurate backoff strategies without hammering downstream services.

How to Prompt for the Full Header Spec

Vague prompts produce vague code. The most effective technique is to paste the exact header contract you want implemented directly into the prompt, then ask ChatGPT to use it as the authoritative spec.

Here's a prompt template that works well:

You are implementing API rate limiting middleware for a FastAPI application.
Use a fixed 60-second window, per-user limit of 100 requests.

Requirements (treat these as the authoritative spec — do not add or omit headers):
1. Set X-RateLimit-Limit on every response (integer, the per-window limit).
2. Set X-RateLimit-Remaining on every response (integer, clamped to 0 on 429).
3. Set X-RateLimit-Reset on every response (Unix epoch integer, seconds precision).
4. Set X-RateLimit-Window on every response (integer seconds, here: 60).
5. On 429 responses only, set Retry-After to the integer seconds until the window resets.
6. Return 429 with a JSON body: {"error": "rate_limit_exceeded", "retry_after": <seconds>}.

User identity comes from a verified JWT sub claim available as request.state.user_id.
Storage backend: Redis. Use aioredis with async/await.

After the middleware, provide a separate Python snippet showing a client
that reads X-RateLimit-Remaining and Retry-After and backs off correctly.
Do not skip the client snippet.

Three things make this prompt work. First, it hands ChatGPT a numbered spec, so there's a concrete checklist to satisfy. Second, it names the exact storage backend and auth mechanism, which eliminates the guessing that leads to placeholder comments. Third, it explicitly demands the client snippet and warns against skipping it — because ChatGPT will skip it if you don't say that.

The output you'll get from this prompt should look roughly like the following middleware:

import time
import aioredis
from fastapi import Request, Response
from fastapi.responses import JSONResponse
from starlette.middleware.base import BaseHTTPMiddleware

RATE_LIMIT = 100
WINDOW_SECONDS = 60

class RateLimitMiddleware(BaseHTTPMiddleware):
    def __init__(self, app, redis_url: str):
        super().__init__(app)
        self.redis_url = redis_url
        self.redis = None

    async def dispatch(self, request: Request, call_next):
        if self.redis is None:
            self.redis = await aioredis.from_url(self.redis_url)

        user_id = getattr(request.state, "user_id", "anonymous")
        now = int(time.time())
        window_start = now - (now % WINDOW_SECONDS)
        window_reset = window_start + WINDOW_SECONDS
        key = f"rl:{user_id}:{window_start}"

        count = await self.redis.incr(key)
        if count == 1:
            await self.redis.expire(key, WINDOW_SECONDS + 5)

        remaining = max(0, RATE_LIMIT - count)
        headers = {
            "X-RateLimit-Limit": str(RATE_LIMIT),
            "X-RateLimit-Remaining": str(remaining),
            "X-RateLimit-Reset": str(window_reset),
            "X-RateLimit-Window": str(WINDOW_SECONDS),
        }

        if count > RATE_LIMIT:
            retry_after = window_reset - now
            headers["Retry-After"] = str(retry_after)
            return JSONResponse(
                status_code=429,
                content={"error": "rate_limit_exceeded", "retry_after": retry_after},
                headers=headers,
            )

        response = await call_next(request)
        for k, v in headers.items():
            response.headers[k] = v
        return response

Notice the Redis key includes window_start, not just the current minute. This is a fixed-window approach. If you want a sliding window, say so explicitly in the prompt; ChatGPT will default to fixed-window unless told otherwise.

Handling the Retry-After Header Correctly

RFC 7231 allows Retry-After to be either delta-seconds (an integer like 42) or an HTTP-date (like Wed, 21 Oct 2025 07:28:00 GMT). ChatGPT sometimes generates the HTTP-date format when you haven't specified, which breaks clients that only parse delta-seconds.

Lock the format in your prompt. For most APIs, delta-seconds is the right choice because it's simpler to parse and doesn't require clock synchronization between server and client. Add this line to your prompt: