Skip to content

Pagination in the HTTP Request node#

The HTTP Request node supports pagination. This page provides some example configurations, including using the HTTP node variables.

Refer to HTTP Request for more information on the node.

API differences

Different APIs implement pagination in different ways. Check the API documentation for the API you're using for details. You need to find out things like:

  • Does the API provide the URL for the next page?
  • Are there API-specific limits on page size or page number?
  • The structure of the data that the API returns.

An example API#

You can try these examples with any API that supports pagination. If you need a free option, Punk API is free with rate limits, and doesn't require authorization.

Enable pagination#

In the HTTP Request node, select Add Option > Pagination.

Use a URL from the response to get the next page using $response#

If the API returns the URL of the next page in its response:

  1. Set Pagination Mode to Response Contains Next URL. Ensemble displays the parameters for this option.
  2. In Next URL, use an expression to set the URL. The exact expression depends on the data returned by your API. For example, if the API includes a parameter called next-page in the response body: 
    1
    {{ $response.body["next-page"] }}

Get the next page by number using $pageCount#

If the API you're using allows you to target a particular page by number:

  1. Set Pagination Mode to Update a Parameter in Each Request.
  2. Set Type to Query.
  3. Enter the Name of the query parameter. This depends on your API. For example, Punk API uses a query parameter named page to set the page. So Name would be page.
  4. Hover over Value and toggle Expression on.
  5. Enter {{ $pageCount + 1 }}

$pageCount is the number of pages the HTTP Request node has fetched. It starts at zero. Most API pagination counts from one (the first page is page one). So adding +1 to $pageCount means the node fetches page one on its first loop, page two on its second, and so on.

Set the page size in the query#

If the API you're using supports choosing the page size in the query:

  1. Select Add Parameter in Query Parameters. This is in the main node parameters, not in the option settings.
  2. Enter the Name of the query parameter. This depends on your API. For example, Punk API uses a query parameter named per_page to set page size. So Name would be per_page.
  3. In Value, enter your page size.