Why Webhook Events Arrive Out of Order

A webhook sends a notice from one service to another when something changes, such as a customer record or an order. Events arrive out of order when the receiver gets or applies a notice about an earlier change after a notice about a later change. The receiver’s stored data can then stop matching the provider’s current state.

A provider can reorder events when it retries delivery, and a receiver can reorder them when it uses a queue and workers. A retry delays another attempt to deliver an older event. A queue holds events so the receiver can handle them later. Workers are the code that handles queued events; when several workers run at once, they can finish jobs in a different order from the order they started. To find where the order changed, compare when the change occurred, when its event reached the receiver, and when the receiver finished processing it.

Retries let newer events overtake earlier ones

A provider’s retry can put an old event behind a newer one. Suppose the same record changes twice, producing events A and B. The provider sends A, but the receiver times out or returns an error. While the provider schedules another attempt to send A, it sends B successfully. The retry for A then arrives after B.

Houzz Pro describes this sequence directly: it dispatches events for the same resource in parallel, and a delayed retry of an earlier event can arrive after a later update. A retry is another delivery attempt for the same event; it does not make that event newer. The provider also documents at-least-once delivery, which means the receiver can get the same event more than once. (Houzz Pro’s webhook documentation)

Retry policies vary by provider. A provider might retry automatically, offer manual redelivery, or stop attempting delivery after a limit. A failure therefore does not have one universal outcome, and retry behavior should not be assumed from another provider’s policy.

Queues and workers create another ordering boundary

A queue lets a receiver confirm that it got a webhook and handle the event later. This separates incoming requests from slower work, but it does not guarantee that events enter the queue in the order the changes occurred.

Even when the receiver adds events to the queue in arrival order, workers running at the same time can finish them in a different order. For example, worker one starts A and waits for a slow service, while worker two starts B and finishes quickly. The receiver can then apply B before A. If both events update the same record, applying A last can overwrite the newer data.

Amazon SQS standard queues may deliver messages out of order. SQS FIFO queues preserve message order within each message group, while allowing messages in different groups to be handled at the same time. These guarantees cover messages already in the queue. They do not ensure that the webhook provider sent events in the order the changes occurred. (SQS standard queue behavior and FIFO delivery logic)

To find where events changed order, record each event’s identifier and, when available, the timestamp the provider assigned to it. Also record when the receiver accepted the event and when it finished processing it. Comparing these times helps show whether the provider or the receiver changed the order.

Providers do not share one ordering guarantee

Several providers explicitly warn that webhook events can arrive out of order, but their retry and recovery policies differ. For example:

These policies vary by provider, so check the documentation for the webhook subscription you use. Find out whether the provider guarantees event order. Check what happens when delivery fails: does the provider retry automatically, or can you resend an event manually?

Timestamps and event IDs answer different questions

A timestamp tells you when the provider says a change occurred, but it does not always show which of two events came first. Stripe’s snapshot events use timestamps that are precise to the second, so separate events can have the same timestamp. Shopify recommends timestamps as a way to organize events, but timestamps alone do not guarantee event order.

An event ID identifies an event or delivery, but it does not tell the receiver which of two different events is newer. Stripe recommends using event IDs to recognize duplicate events. If an event describes a change but does not provide enough information to determine the resource’s current state, Stripe recommends retrieving the related object through its API. (Stripe’s event-handling guidance)

To get the latest state, the receiver should use ordering information the provider guarantees, such as a resource version or sequence number, if available. If the provider supplies no reliable sequence, treat the event as a signal to check the resource’s current state, not as a complete record of changes in time order. Before relying on timestamps or event IDs, confirm what they mean and how the provider lets you recover the current state.

What I'm building

Delegate tasks. Get software.

Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.

Take a look at vroni.com