Isolating Webhook Failures: Provider Retries vs. Receiver Bugs

When a provider sends your service a webhook, an HTTP request that carries an event, it expects an HTTP response. The response’s numeric status code tells the provider how the request went. A response in the 2xx range generally marks the delivery as successful. If the provider receives a different status code or no response before its deadline, its rules determine whether it marks the attempt as failed and retries.

Start by checking the provider’s documentation and delivery log. The status code, error details, and timing show whether the receiver returned an error, the request timed out, or the connection failed before the receiver could answer. Use those clues to tell a receiver bug from a network problem or a provider-side issue.

Read the provider’s delivery rules first

Providers set their own timeout limits and retry rules. Check which status codes trigger retries and whether redirects count as failures. Also find out how long the provider waits for a connection or response. A connect timeout limits the time to establish a connection; a read timeout limits the wait for a response after connection. A total timeout sets an overall deadline.

For example, Twilio’s connection override documentation lists default connection, read, and total timeout settings. Twilio also documents retry conditions that can include connection or secure-connection failures, read timeouts, and selected HTTP status codes. Its voice-call webhooks have a separate hard timeout limit. These details apply to Twilio’s documented products and settings; another provider’s limits may differ.

For each attempt, the provider’s delivery log shows when it occurred, along with the HTTP status and error name. It also records the retry count and any response body. Hook0’s delivery-log guidance describes these fields, including connection errors that appear without an HTTP status. Compare the log with your receiver’s access and application logs for the same time.

Follow the status and timing clues

Use these clues and apply the provider’s documented rules to each one:

1. The provider recorded a 2xx response. The receiver acknowledged the request, so the provider generally treats the delivery as successful. If the expected work did not happen, check what the receiver did after acknowledging the request. For example, see whether a queued task or another service the receiver called failed.

2. The provider recorded a 3xx, 4xx, or 5xx response. The provider received an HTTP response. A 3xx status redirects the request; some providers treat redirects as failures. Svix, for example, retries non-2xx responses, which includes redirects. A 4xx response means the receiver or another server rejected the request. The webhook URL might point to a route that is unavailable, or the receiver might have rejected the request during validation. A 5xx response means a server handling the request reported an error. Check the receiver and any proxy or gateway in front of it. The Oracle status-code documentation describes these broad status categories.

   Treat 408 and 429 as special cases when diagnosing 4xx responses. A 408 means the receiver waited too long for the request. Compare the provider’s timeout rules with the receiver’s logs to find where the delay occurred. A 429 means the receiver is rate limiting requests. Check whether the response includes Retry-After, which tells the sender when to try again, and whether the provider honors it. Retry behavior still depends on the provider’s policy.

3. The provider recorded no HTTP status and a connection error. The provider did not receive an HTTP response. A failed DNS lookup (the step that finds a server’s address from its domain name) points to the connection path. A refused connection or failed secure-connection setup points there too. They do not identify which party caused the problem. Compare the provider’s attempt time with the receiver’s access logs, whether the webhook URL was available, and any network or DNS changes. If the receiver logged the request but did not answer before the deadline, investigate slow receiver code or an overloaded dependency. If the receiver logged no request, check whether the domain name resolved to an address, whether the network routed the request, and whether the connection was established. Missing receiver logs alone do not prove a provider fault.

4. The provider recorded a timeout. Compare the elapsed time with its connect, read, and total limits. A failure when the connection deadline expires means the provider could not establish the connection in time. A failure after the receiver accepted the request means the response was delayed somewhere between the receiver and provider. The receiver’s logs help distinguish slow handling from a delay between the receiver and provider. The webhook retry guidance from Hookdeck also treats connection failures and timeouts as different failure signals.

Use retry patterns to narrow the cause

A retry log shows what the provider saw, but it does not always show what caused the failure. Repeated 5xx responses from one endpoint that match receiver errors support a receiver-side diagnosis. When several unrelated endpoints show connection errors or timeouts at the same time, a shared provider or network issue is more plausible. Confirm that pattern against provider status information and your own network records before assigning the cause.

A provider’s retry schedule also affects what the logs show. Some providers retry only selected errors; others retry all non-2xx responses. Svix documents eight attempts over roughly 24 hours with increasing delays between attempts. That schedule applies to Svix. Check the relevant provider’s documentation to see whether an event is still being retried or has reached the retry limit. After retries end, check whether the provider offers a dead-letter queue, where undelivered events wait for inspection or replay.

What I'm building

Delegate tasks. Get software.

Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.

Take a look at vroni.com