Trusting X-Forwarded-For Is How Rate Limiting Quietly Stops Working

Almost every web application eventually needs to know who a request came from. Rate limiting needs it, audit logs need it, abuse detection needs it, and behind a load balancer or a CDN the socket’s peer address is your proxy, not your user. So you reach for X-Forwarded-For.

The default way to read that header hands the client control of its own identity, which means anything you built on top of it is decorative.

What the header actually is

X-Forwarded-For is a comma-separated list that grows left to right as a request passes through proxies:

X-Forwarded-For: 203.0.113.7, 198.51.100.4, 10.0.0.9

The convention is that the leftmost entry is the original client and each proxy appends the address it received the request from. Which sounds like the leftmost value is what you want:

def client_ip(request):
    xff = request.headers.get("x-forwarded-for", "")
    return xff.split(",")[0].strip() or request.client.host

That code is everywhere. It’s in tutorials, it’s in Stack Overflow answers, and it’s in a lot of production middleware.

The problem is that your proxies append, they do not verify. If the client sends a request that already contains an X-Forwarded-For header, your proxy appends to it rather than replacing it. So a client can put anything it likes at the front of that list, and your leftmost read returns whatever the client typed.

What that costs you

Concretely, rate limiting stops working. Not degrades, stops:

for i in range(200):
    requests.get(url, headers={"X-Forwarded-For": f"1.2.3.{i % 256}"})

Each request appears to come from a different client, so each gets its own bucket, and nothing is ever throttled. I have a regression test for this exact scenario, and the number is stark: with a trusted-hop implementation the limiter blocks 45 of those requests, and with a leftmost read it blocks 0.

The audit log fails the same way, and worse, because it fails silently and permanently. Every entry records an attacker-chosen address, so the record you’ll want during an incident is fiction, and you won’t discover that until the incident. Password reset throttling, login attempt counting, and IP-based blocklists all inherit the same hole.

Counting hops from the right

The correct model starts from a different question. Instead of “who does the client claim to be,” ask “which entries in this list were written by infrastructure I control.”

Your proxies are the rightmost entries, because each one appended as the request came through. If you sit behind exactly one proxy, then that proxy appended one entry, and the address it saw is the last entry in the list. Everything to the left of that was supplied by the client and is unverifiable.

So the rule is: skip N entries from the right, where N is the number of trusted proxies between you and the internet, and take the next one.

def client_ip(request, trusted_hops: int = 1) -> str:
    peer = request.client.host
    raw = request.headers.get("x-forwarded-for", "")
    chain = [p.strip() for p in raw.split(",") if p.strip()]

    # Each trusted proxy appended one entry. The address the outermost
    # trusted proxy observed is at index -trusted_hops.
    if trusted_hops < 1 or len(chain) < trusted_hops:
        return peer                       # absent or too-short chain
    candidate = chain[-trusted_hops]

    try:
        ip = ipaddress.ip_address(candidate)
    except ValueError:
        return peer                       # malformed entry
    return str(ip)

Three properties of that function matter as much as the indexing:

The hop count is configuration, not a constant. It’s a property of your deployment: one for a single load balancer, two behind a CDN in front of a load balancer. Default it to 1, make it an environment variable, and write down what your topology is in a comment. Getting the number wrong in the safe direction (too few hops) means you attribute traffic to your own proxy and over-throttle. Wrong in the unsafe direction means you’re back to trusting the client.

Validate that the result is an IP. The value is a string from a header, so it can be anything. Parsing it with ipaddress also normalizes representations, so the same client can’t occupy several buckets by varying the spelling of an IPv6 address.

Fall back to the peer address, always. No header, too few entries, unparseable entry: all of these fall back to the socket peer, which is the one address nobody can forge. A missing header should never produce None flowing into a dictionary key, because “the client with no IP” becomes a shared bucket that everybody can hide in.

And the same treatment applies to X-Real-IP, which is exactly as forgeable and which people often trust unconditionally because it looks singular and official.

One helper, every consumer

The part that took me longest to get right had nothing to do with the parsing. It was that I had three implementations.

Rate limiting read the header one way. The audit log read it another. Session records read the peer address directly. So the same request produced three different identities, and the audit trail disagreed with the limiter about who had done what. That kind of disagreement is worse than either implementation being wrong, because it makes the record uninterpretable.

There is now exactly one client_ip() helper, and rate limiting, audit logging, password reset throttling, and session records all call it. That’s the real deliverable: not the parsing logic, the single source of it.

Verify it with a mutation

This is a change where you should not trust the code review. Write the rotating-header test, run it against the fixed implementation, then put the leftmost read back and confirm the test fails:

FAIL: rotating XFF: 0 requests blocked, expected 45

Then restore the fix. That thirty-second loop is the difference between believing your limiter works and knowing it.

Three rules to hold

Client IP is derived from your deployment topology, not from a header. Three rules:

1. Never read the leftmost X-Forwarded-For value. Count trusted hops from the right.
2. Make the hop count configuration, default it to 1, and document your topology next to it.
3. Have one helper, and make every consumer use it, so your limiter and your audit log can never disagree about who someone was.