Handling Paginated API Responses with Laravel HTTP Client

When a Laravel service asks an external API for many records, the API may split them into smaller groups called pages. Laravel’s HTTP client sends each request and reads each response. The API specifies how to request the next page and how to tell when it has sent all the results.

For an API that uses numbered pages, your code can increment a page query parameter, append each response’s items to a collection, and stop when the API reports the last page. Other APIs use a next-page URL or a cursor, which is a value that marks where the next request should continue. Check the API’s pagination rules before choosing the request parameter or stopping condition.

Loop through numbered pages

Laravel’s HTTP client takes query parameters as the second argument to get(). It can also ask for a JSON response, attach a bearer token (a credential that authenticates the request), set a timeout, and read the response with json(). See the Laravel HTTP Client documentation for details.

The following example assumes the API accepts page and per_page parameters and returns items in data with the final page number in meta.last_page. These field names and response shapes belong to this example’s API contract; Laravel does not require them.

use Illuminate\Support\Facades\Http;
use RuntimeException;
use UnexpectedValueException;

$page = 1;
$maxPages = 5000;
$results = [];

while (true) {
    if ($page > $maxPages) {
        throw new RuntimeException(
            "Pagination exceeded the safety limit of {$maxPages} pages."
        );
    }

    $response = Http::acceptJson()
        ->withToken($token)
        ->timeout(20)
        ->get($endpoint, array_merge($filters, [
            'page' => $page,
            'per_page' => 100,
        ]))
        ->throw();

    $payload = $response->json();
    $items = is_array($payload) ? ($payload['data'] ?? null) : null;
    $lastPageValue = is_array($payload)
        ? ($payload['meta']['last_page'] ?? null)
        : null;

    if (
        ! is_array($items)
        || filter_var($lastPageValue, FILTER_VALIDATE_INT) === false
    ) {
        throw new UnexpectedValueException(
            'The API returned an unexpected pagination response.'
        );
    }

    $lastPage = (int) $lastPageValue;

    if ($lastPage = $lastPage) {
        break;
    }

    $page++;
}

The loop starts at page 1, sends the request, and adds that page’s items to $results. It stops when the requested page reaches the final page reported by the API. The page limit prevents the loop from sending requests without end if the API returns unexpected page information. If the loop reaches the limit, it throws an exception instead of presenting partial results as complete.

The 20-second timeout, page size of 100, and limit of 5000 pages are example settings, not universal requirements. Choose values that fit the API’s documented limits and the needs of your service.

Use the API’s completion signal

Use the clearest completion signal described in the API’s documentation. A response might include a has_more flag, a last-page number, or a link to the next page. For example, GitHub’s REST API uses a Link response header with a rel="next" link when another page is available. This describes GitHub’s API; other providers define their own pagination formats.

For a next-page URL, request the supplied link and stop when the API omits it. Laravel’s response object exposes headers, but Laravel does not provide a universal parser for pagination links. Check the target API’s documentation for the link format and authentication requirements before following it.

An empty or short page is a valid stopping signal only when the API says it is. Some APIs allow pages with fewer items even when more results remain. Stopping on page length alone in that case silently drops records.

Handle failed and large requests

Laravel’s HTTP client returns responses with unsuccessful status codes unless you tell it to throw an exception. Calling throw() makes an unsuccessful response raise an exception, so the loop does not treat an error response as valid data. Laravel documents a default response timeout of 30 seconds and a default connection timeout of 10 seconds. Set explicit limits if those defaults do not fit the external API’s response behavior.

Appending every item to $results keeps the full result set in memory. That works when the complete set is reasonably sized. For larger sets, process or save each page inside the loop, then discard its items before fetching the next page.

Test the page sequence

Laravel’s HTTP testing tools can fake outgoing requests and return responses in sequence. Use them to check that the client requests page 1 and the expected later pages, collects items from each response, and stops at the completion signal declared by the API. Also test an unsuccessful response and invalid pagination details. These tests show whether the code fails clearly instead of returning an incomplete collection. The Laravel HTTP testing documentation describes request fakes and response sequences.

What I'm building

Delegate tasks. Get software.

Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.

Take a look at vroni.com