Currently Available: Need a skilled Software Developer for your next project?
Categories
APIs

API Design: Pagination and Webhook Contracts

Define pagination parameters and response metadata

Every collection endpoint needs a clear way to request another page and tell when the collection ends. Offset pagination uses a starting position, such as offset=100&limit=50, so clients can jump to a position. It fits smaller, relatively stable collections and interfaces that need direct page access. Inserts or deletions before that position shift later results, which can cause clients to see duplicates or skip records.

Cursor pagination uses a continuation value that marks where the server should resume. It fits large or frequently changing collections, especially when clients read pages in sequence. The server should order results deterministically, such as by creation time and then by a unique ID when timestamps tie. A cursor is opaque: clients pass it back unchanged and do not inspect or construct its contents. The API should also define whether filters and sort options must stay the same between requests. Google’s pagination guidance and Zalando’s API guidelines describe these approaches and their trade-offs.

Choose parameter names and limits once, then document them consistently. Common choices include limit with offset, or page_size with page_token. State the default and maximum page size, and explain how the server handles values outside that range. For example, Google AIP-158 uses a default of 50 and a maximum of 1,000 in its guidance; those figures are examples, not universal requirements. Clients should be able to understand the same parameter rules across endpoints.

Return each collection in the same response structure: an items array for the records and fields that describe the page. Use either a next-page URL or a continuation token, and make the end condition unambiguous: omit the next link or return an empty token when no page remains. Keep field names and behavior consistent across endpoints. If you include a total count, say whether it is exact or estimated; calculating it can add cost.

For example, a cursor-based endpoint might use:

GET /v1/orders?page_size=50&page_token=opaque-token
{
  "items": [
    { "id": "order_123" },
    { "id": "order_124" }
  ],
  "next_page_token": "next-opaque-token"
}

The values are illustrative. The client sends next_page_token unchanged on its next request; an empty value or an omitted field signals that no further page exists.

Offset pagination remains a good fit for small collections where clients need to jump to a page by number. It is simple to implement with limit and offset, but large offsets can slow queries, and inserts or deletes between requests can shift rows, causing clients to miss or repeat results. Cursor pagination supports sequential reads more reliably as collections change, provided the server uses a deterministic sort order.

Make pagination reliable as data changes

A cursor works reliably only when each record has a definite place in the sort order. Sort by a field such as creation time, then add a unique tie-breaker such as the record ID. Without that tie-breaker, records with equal timestamps can move unpredictably between pages, causing clients to see duplicates or miss records.

The cursor should let the server resume at the same position in that ordering. When your contract requires it, the cursor should also preserve the filters and sort options from the first request. The Google pagination guidance requires clients to keep other request arguments consistent while paging, though it allows page size to change. Choose and document the same rule for your API. If a client changes a filter or sort option, return a clear client error or require it to start a new traversal; do not silently apply the changed query to an old cursor.

The server can keep the same sort order while records are inserted, deleted, or updated as a client pages. Later responses can reflect those changes. A record whose sort field changes might appear twice or be skipped, depending on how the cursor works. If a cursor refers to a particular record as its reference point, define what happens when that record is deleted. A cursor based on the sort values and unique ID lets the server continue without looking up that record. Zalando’s pagination guidance discusses cursor behavior and this deleted-anchor failure.

Document whether pages represent a live view or a consistent snapshot. A snapshot keeps the collection fixed across requests, but a cursor alone does not provide that guarantee. If the API does not implement snapshots, state that concurrent changes can affect traversal and that clients needing a complete current inventory should run a reconciliation or start a fresh scan.

Cursors can also become invalid or expire. Return a documented error with a stable code that programs can check. The code should tell clients whether the cursor is malformed, expired, or incompatible with the request. Explain whether the client should retry, restart from the first page, or abandon the traversal. Never treat an invalid cursor as an empty page or silently restart at the beginning, since either behavior can make clients mistake incomplete results for a complete collection. Document any cursor lifetime and which query parameters clients must preserve.

Specify webhook event and delivery contracts

A webhook contract defines what an event means and how a sender delivers it. Give every event a stable identity and a type. Include the time it occurred, a schema version that identifies the payload format, and a payload. Keep the shared fields in the same structure across event types. Put event-specific data in the payload so consumers can read the shared fields before handling a particular event.

For example, a record.updated event might use this shape:

{
  "id": "",
  "type": "record.updated",
  "occurred_at": "",
  "schema_version": "",
  "data": {
    "record_id": "",
    "status": ""
  }
}

Explain whether the event ID stays the same when the sender retries delivery. Consumers need a stable ID to recognize the same event arriving again. A delivery ID identifies a transmission; it may differ from an event ID when the sender creates separate deliveries for the same underlying change.

Define verification and acknowledgment

Specify how the sender signs each request. Identify the bytes it signs and where it puts the signature, then explain how the receiver checks it. For example, GitHub’s webhook guidance uses an HMAC-SHA256 signature, a keyed message-authentication code built with SHA-256, calculated from the raw request body. It recommends comparing signatures in constant time, so the time taken does not reveal whether parts of the signature match. State your own algorithm and encoding explicitly; header names and signature formats vary between providers. Require HTTPS and document how operators store and rotate secrets.

Define which response counts as acknowledgment and the deadline for returning it. The receiver should verify the signature, save the event to storage that survives a crash, and then acknowledge it. After saving it, the receiver can process the event in the background. If it acknowledges before durable storage, a crash can discard an event that the sender considers delivered. Document the timeout and what happens after a timeout or non-success response, since senders use different policies: GitHub does not automatically redeliver failed deliveries, while other providers document automatic retries.

Document retries, duplicates, and order

State how long the sender retries. Specify which responses trigger another attempt and how delays grow between attempts. A policy can use progressively longer delays with a cap, but publish the actual retry window and any manual redelivery option. Tell consumers to expect duplicate deliveries whenever a sender retries or a network failure leaves acknowledgment uncertain.

Consumers should record processed event IDs and make the corresponding business action idempotent, meaning that handling the same event twice produces the same result as handling it once. Do not imply that delivery order matches event order. State whether the sender guarantees ordering and, if so, its scope. If the sender does not guarantee order, include sequence numbers or resource-version values where available, or tell consumers to fetch the current state through the API. GitHub documents out-of-order deliveries, so event timestamps alone should not be presented as an ordering guarantee.

Finally, document breaking payload changes, schema-version support, and deprecation dates or steps. Test signature failures, timeouts, retries, duplicate and out-of-order events, and old and new payload versions. Provide a way to inspect failed deliveries or compare consumer data with the current API state to recover events missed during an outage.

What I'm building

Delegate tasks. Get software.

Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.

Take a look at vroni.com

Email updates

Usually a new article and a few links I found interesting.

No spam. Unsubscribe with one click.

Leave a Reply

Your email address will not be published. Required fields are marked *