How Does Pagination Work In Elasticsearch

If you have ever used Kibana, you will notice that it allows you to view specific information stored in an Elasticsearch index. However, since most indexes contain thousands of records, Kibana trims the number available per page, allowing you to navigate from page to page and view the next or previous records.

In this tutorial, we will learn how to paginate results in Elasticsearch using the pagination APIs.

The following screenshot shows how you can implement pagination fr Elasticsearch for front-end applications.

In Elasticsearch, there are three main ways to perform pagination. Each method has its advantages and disadvantages. It is therefore essential to consider the structure of the data stored in your index.

In this guide, we will learn how to paginate using the three main methods. Namely:

1. From and size pagination
2. Scroll pagination
3. Search after pagination.

From and Size pagination

When you make a search request in Elasticsearch, you will get the top 10 hits of the matching query. If you have a search query that returns more documents, you can use the from and size parameters.

The from parameter is used to define the number of records to skip before displaying the preceding documents. Think of it as the index at which Elasticsearch starts showing the results.

The size parameter will describe the maximum number of records that the search query will return.

The from and size parameters are very applicable when you want to create paged results.

Consider the query below that illustrates how to use the from and size parameters:

In the query above, we search the documents matching specific criteria. We then use the from and size parameters to determine how many records the query will display.

In our example, we start at the first matching documents. i.e., we start at index 0.

We also specify the maximum number of documents to display to 5.

The results from the query are as follows:

As you can see from the response above, we have seven total hits. However, we limit the maximum documents to show as 5.

To view the last two documents, we can set the from value to 5 as:

Scroll Pagination

The next type of pagination in Elasticsearch is scroll pagination. It requires a unique scroll_id that determines the number of documents to show and the duration of the search context.

Consider the documentation to learn more about the search context.

To generate the scroll_id, make a request as shown below:

The query above should return the results, including the scroll_id as shown:

The scroll parameter in the search query tells Elasticsearch to use 1 minute as the duration for the search context.

To use the scroll API and view the following batch of 20 results, use the scroll_id as shown:

The query should return the next batch of documents matching the specified query.

To clear a scroll, use a delete request as:

The request should remove the scroll as specified by the id. It is good to note that the search context is cleared automatically when the set duration expires.

Search after pagination

The other pagination method in Elasticsearch is search_after. The idea behind search_after is to retrieve values after a sort value.

Let us take a simple example. Suppose we want to view the documents the DestCityName = Denver and sort based on the ticket price.

If we run the above query, we should see only two out of the total hits, as specified by the size parameter.

It will also provide us with a sort value for every document as shown:

We can use this sort value to fetch the next batch of documents as:

We then use the search_after parameter and the sort id provided in the last request to view the next batch of documents.

Closing

This guide gives you the basics of paginating results in Elasticsearch using from and size pagination, scroll, and search_after pagination. Consider the documentation to explore.