Retries, timeouts and idempotency: the parts of an SDK nobody hand-writes
Every client library needs them and almost no hand-written one has all three. The defaults matter more than the knobs, because most consumers never touch the knobs.
Look at a hand-written client library and you can usually date it by what it does about failure.
The first version has none of this. Someone adds a retry loop after an outage. Someone adds a timeout after a hang. Nobody adds jitter, because the thundering herd only shows up at a scale you do not have yet, and nobody adds idempotency keys, because that requires the server to cooperate and by then the client is somebody else's file.
These three are the least interesting code in any SDK and the most consequential, and they are the clearest argument for generating clients rather than writing them: you get to make the decision once, correctly, and every language ships it.
Retries: the status codes are the hard part
The mechanism is easy. Knowing what is safe to repeat is not.
A sensible default retries on the statuses that mean "try again", which is a smaller set than people assume: 408, 425, 429, and the 5xx family, 500, 502, 503, 504. It does not retry a 400, because sending the same malformed request again produces the same malformed response, slower.
The sharper distinction is by method. A GET is safe to retry by definition. A POST is not, because a 5xx does not tell you whether the server processed the request before it failed. A client that blindly retries POST on a 500 is a client that occasionally creates two of something.
So the safe rule is: retry idempotent methods on the full retryable set, and retry non-idempotent methods only on the statuses that explicitly mean the request was not processed, which in practice is 429 and 503. Both of those are the server saying "I did not do this, come back". Everything else in the 5xx range is ambiguous, and ambiguity plus a write is a duplicate.
Backoff needs jitter, and jitter is the part that gets dropped
Exponential backoff is well known. Jitter is the half that matters and the half that gets skipped.
Without it, every client that failed at the same moment retries at the same moment, then again together, and again. Your outage recovers into a synchronised wall of traffic from your own SDK. The failure mode only appears once you have enough consumers to notice, which is exactly when you can least afford it.
Reasonable defaults: three total attempts, a first retry around 500ms, a ceiling around 8 seconds, and randomisation across the whole delay rather than a small wobble around it. Three attempts is deliberately modest. Retries are a way to survive a blip, not a way to survive an outage, and a client that tries ten times moves the user's failure ten seconds later while adding load to a server that is already struggling.
Timeouts are per attempt, not per call
This one is quietly wrong in a lot of clients.
If you set a 30-second timeout and allow three attempts, a caller can wait 90 seconds plus backoff before seeing an error. Somebody's request handler is sitting behind that, holding a connection, and their own upstream timeout fires first, which means your carefully designed retry produced a worse outcome than failing fast.
Be explicit about which one you mean. A per-attempt timeout with a bounded attempt count gives a predictable worst case, and that worst case belongs in your documentation so a caller can set their own budget above it.
Idempotency is a contract, not a client feature
A client can generate a key. Only the server can honour it.
The pattern: the client generates a unique key per logical operation, sends it as a header on writes, and reuses the same key when retrying that operation. The server records the key with the result and returns the recorded result for a repeat instead of performing the work again.
This is the mechanism that makes retrying a POST safe, and it is the reason it is worth the coordination. Without it, every write is a coin flip on ambiguity. With it, a client can retry a payment on a 503 and be certain it will not charge twice.
Two design details decide whether it works:
Scope the key to the operation, not the attempt. The whole point is that all attempts of one logical action share a key. A key regenerated per HTTP request is an expensive random header.
Decide how long you remember. Twenty-four hours is common. Whatever you pick, publish it, because it is the window in which a client can safely retry, and a client that retries outside it gets a duplicate with no warning.
The methods that need it are the ones that create or mutate: POST and PATCH. PUT and DELETE should already be idempotent by their own semantics, and if yours are not, that is worth fixing before adding a header.
Defaults matter more than knobs
Every one of these should be configurable, and almost nobody will configure them.
That is not a criticism of your users. Someone integrating your API is solving their problem, not tuning your client. Whatever ships as the default is what runs in production for the overwhelming majority of your consumers, forever.
Which means the useful question about a client library is not "can I set the retry policy". It is "what does it do if I never open the settings". A client that requires configuration to behave well behaves badly, because the configuration step is the one nobody does.
Configuration earns its place at the edges. An endpoint that is expensive and safe wants a longer timeout and more attempts. An endpoint that triggers something irreversible wants no retries at all regardless of status. Those are per-endpoint decisions and they should be expressible per endpoint, on top of a project-wide default that is already correct.
What this looks like generated
Every client we generate ships with all three wired in and consistent across languages: backoff with jitter on the retryable statuses, a per-attempt timeout with a bounded attempt count, and idempotency keys on the methods that need them. The project-wide policy is a setting, and any single endpoint can override it when its risk profile differs.
The reason to do it at the generator rather than per language is that these are the details that drift. A team that hand-writes six SDKs writes six retry policies, and by the second year they disagree about which statuses are retryable. One policy, six emissions, no drift.
If you want the failure side of this rather than the prevention side, why generated SDKs break silently covers what happens when a client fails in someone else's production. And declaring your failure responses is one of the rules in our spec audit, because a client can only offer typed errors for the failures your spec admits to.
