externalTrafficPolicy Local: client IP for skewed load
The setting that preserves the source address also hands each node an equal share of traffic regardless of how many pods it is running
Every access log said 10.244.1.x. Rate limiting was useless, the geo rules did nothing, and the abuse dashboard showed one very busy internal address.
The fix is one line, and everyone finds it within an afternoon:
spec:
externalTrafficPolicy: LocalReal client addresses come back immediately. What arrives later is a load distribution nobody asked for.
Why the addresses were wrong
Default policy is Cluster. Traffic hits any node, and kube-proxy forwards it to a pod wherever that pod lives.
That forward is the problem. If the packet arrives at node A and the pod is on node B, node B's reply has to travel back through node A, because node A is the one holding the connection with the load balancer. To make the return path work, node A rewrites the source address to its own. Your pod sees the node, not the client.
Local tells kube-proxy to stop forwarding. If there's a pod for this Service on the node the packet landed on, deliver it locally, unmodified. If there isn't, drop it. No cross-node hop means no return-path problem means no rewrite, and the client address survives.
The dropped-packet half sounds alarming and mostly isn't, because cloud load balancers health-check the NodePort. Nodes without a local pod fail the check and stop receiving traffic. The mechanism is doing exactly what it says.
The skew
Here's what the health check can't fix.
The load balancer distributes across the nodes that pass the check, and it distributes evenly, because it has no idea how many pods each one is running. It's balancing nodes. You care about pods.
Three nodes pass the health check. Node A runs one pod, node B runs one, node C runs four. The load balancer sends a third of the traffic to each node.
Node A's single pod gets 33% of all traffic. Each of node C's four pods gets about 8%.
A four-to-one spread across identical replicas, from a scheduler decision nobody made deliberately. It shows up as some pods pinned at high CPU while others idle, an HPA that scales on average utilisation and therefore scales at the wrong time, and p99 that tracks whichever pod drew the short straw.
Nothing is broken. Every component is doing its job. The mismatch is that one layer balances nodes and the layer you care about is pods.
Making the distribution match
Two shapes work, and both come down to making pods-per-node uniform.
A DaemonSet gives you exactly one pod per eligible node by construction, which makes node-balancing and pod-balancing the same thing. This is why ingress controllers are so often deployed as DaemonSets, and it's the cleanest answer if the workload is a proxy tier you want on every node anyway.
Otherwise, topologySpreadConstraints with maxSkew: 1 over kubernetes.io/hostname:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: webDoNotSchedule rather than ScheduleAnyway matters here. The soft version lets the scheduler pile pods onto one node when things get tight, which is precisely the state you're trying to prevent.
Both of these have a cost the docs are quiet about: you've coupled your replica count to your node count. Scaling to 7 replicas across 3 nodes gives you 3/2/2 and a 50% spread that no constraint can remove.
The feature that is not the answer
Kubernetes has grown a newer field that comes up in every discussion of this, and it solves a different problem.
spec.trafficDistribution takes PreferSameZone or, since 1.35, PreferSameNode. The older PreferClose is still accepted and is now explicitly deprecated in the source in favour of PreferSameZone, which means the same thing with a name that says what it does.
It's tempting to read PreferSameNode as a gentler Local. It isn't, for two reasons.
It doesn't preserve the client address. trafficDistribution influences which endpoint kube-proxy picks. It has nothing to say about SNAT, and for NodePort and LoadBalancer traffic the client address still gets rewritten. If source IP is what you came for, this field does not deliver it.
And it's a hint. The API comment is unambiguous: implementations "can use this field as a hint, but are not required to guarantee strict adherence." Local is a rule with a defined failure mode, which is what you want from something load-bearing.
What trafficDistribution does help with is cross-zone data transfer cost and the latency of an extra network hop. Worth having. Different problem.
The limits worth knowing
There's a cheaper answer that people skip past because it feels like a workaround. If your traffic is HTTP and it comes through a layer-7 load balancer or an ingress controller, the client address is already in X-Forwarded-For, and reading it there costs you nothing in distribution. Local earns its keep when you're terminating TCP or TLS yourself, or when you need the address for something below HTTP.
Some clouds also expose a middle path. AWS NLB and the ALB controller can target pod IPs directly rather than NodePorts, which sidesteps the node-level hop entirely and gets you both properties at once. If you're on EKS, check that before reaching for Local.
And a detail worth knowing before you debug this at 3am: pods on a node with Local set can reach the Service through the ClusterIP normally. The policy governs traffic entering from outside. Testing from inside the cluster will not reproduce any of the behaviour described here.