Currently Available: Need a skilled Software Developer for your next project?
Categories
APIs

Why Webhook Retries Can Overload Struggling Receivers

A webhook is an HTTP request that one service sends to another when an event occurs, such as a payment update or a change to a record. The receiving service sends an HTTP response so the sender knows whether it has accepted the delivery. If the sender does not receive the response it expects, its rules may tell it to send the same event again.

A retry helps after a brief failure. But when the receiver is already short on capacity, it takes longer to handle requests. The sender's repeated attempts add requests while the receiver handles new events and earlier work. The receiver's limited capacity causes delays and timeouts. Those timeouts trigger retries, which add more requests and put further pressure on the receiver. Each provider sets its own rules, so the receiver must respond and handle events in ways that meet those delivery requirements.

How a failed delivery becomes another request

The sender checks the response status and how long the request takes against its own delivery rules. A connection failure or timeout can trigger another attempt. So can a response the sender treats as unsuccessful. Shopify, for example, treats responses outside the 200 range as errors for HTTPS webhooks and sets a five-second overall request timeout. PayPal’s REST webhook service accepts 2xx responses and documents retries after certain unsuccessful responses or connection failures. These rules apply to those providers, not to every webhook service. See Shopify’s delivery guidance and PayPal’s webhook documentation.

A timeout does not tell the sender whether the receiver completed the work. The receiver may finish the work but miss the deadline for returning its response, or the response may fail to reach the sender. The sender can then repeat an event the receiver has already handled. Providers describe webhook delivery as potentially repeated; for example, Shopify recommends detecting duplicate deliveries.

Why retries add pressure during an outage

A receiver under strain takes longer to handle requests. When it misses the sender’s deadline, the sender may schedule another attempt. That new request adds work for the receiver, which may still be processing the original request, while new events continue to arrive. Whether attempts overlap depends on the sender’s schedule and what happens to the original request after a timeout; overlap is not guaranteed.

When limited resources slow the receiver, each retry uses capacity the receiver needs for other requests. That can make responses slower, leading to more failures and retries. The feedback loop stops when delivery succeeds, the provider exhausts its retries, or another provider rule ends delivery. AWS describes this general retry-amplification risk in its reliability guidance. That guidance supports the mechanism, but it does not measure a universal increase in load for webhook receivers.

A delay between attempts changes when requests arrive; it does not remove the extra work. Shopify, for example, says the interval before a retry increases after a failed delivery. Backoff spaces out requests, but failed deliveries still arrive alongside new events. If the receiver’s capacity stays below incoming demand, those delayed attempts can extend the period of pressure.

What the receiver promises with its response

A success response tells the sender to treat the delivery as accepted under its rules. If the receiver responds before it has safely stored the event for later processing, a later failure can leave the sender believing no retry is needed even though the receiver lost the event. If the receiver keeps the connection open while it does slow business work, it can miss the sender’s deadline and receive a duplicate attempt.

A receiver can separate intake from slower work by placing the event in a queue and responding after the intake path has accepted it. A queue holds work for another part of the service to process later. Shopify recommends queueing webhook payloads to meet its response window, and GitHub recommends a queue when needed to meet its ten-second response limit. The receiver still needs to decide what “accepted” means in its own design: for example, whether the event has been saved or queued securely enough to survive a later processing failure. Provider guidance supports quick responses and later processing, but does not set one universal rule for how securely the receiver must store an event.

The receiver also needs to handle duplicate attempts safely. Idempotent processing means that handling the same event more than once produces the same intended result as handling it once. Shopify recommends idempotent handling or checking its X-Shopify-Webhook-Id header to identify repeated deliveries. Deduplication can prevent repeated business actions, but the receiver still spends resources receiving, validating, and recording each request.

Retry rules differ by provider

Webhook providers do not share one retry schedule. PayPal documents up to 25 delivery attempts over three days for certain failed REST webhook deliveries. GitHub.com, by contrast, says it does not automatically redeliver failed webhook deliveries; an operator must request redelivery manually or use a script. Those differences affect how long a receiver may continue to see attempts and what recovery work it must handle.

Retries improve the chance that a sender delivers an event after a temporary failure, but they do not guarantee delivery. Providers can stop after a retry window or require manual redelivery. A receiver should therefore know the provider’s success statuses, timeout, retry conditions, retry period, and failure behavior. Those details define when the sender stops trying and what the receiver must do to detect and recover from missed events.

What I'm building

Delegate tasks. Get software.

Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.

Take a look at vroni.com

Email updates

Usually a new article and a few links I found interesting.

No spam. Unsubscribe with one click.

Leave a Reply

Your email address will not be published. Required fields are marked *