Elasticsearch is a powerful search and analytics engine that powers large-scale applications such as logging, search engines, and etc.
When working in development mode, where you might have a small database for testing purposes, you may encounter an instance where you must fetch all the records from the database.
In this post, you will learn how to use the Elasticsearch search query to fetch all the records from a given index.
Using Search and Wildcard Character
We can use the Elasticsearch search query and the wildcard parameter to fetch all the records from a given index. The syntax is as shown:
For example, suppose we have an index called Disney holding all the Disney Movies and TV shows records. If we wish to retrieve the first ten records from the index using the search query, we can run the request as shown below:
Running the request above should return the queried documents. An example output is as shown:
"took": 12,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1450,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "disney",
"_id": "5V_eAYMB9kKbW3vlFVrc",
"_score": 1,
"_source": {
"duration": "23 min",
"listed_in": "Animation, Family",
"cast": "Chris Diamantopoulos, Tony Anselmo, Tress MacNeille, Bill Farmer, Russi Taylor, Corey Burton",
"date_added": "November 26, 2021",
"show_id": "s1",
"director": "Alonso Ramirez Ramos, Dave Wasson",
"release_year": 2016,
"rating": "TV-G",
"description": "Join Mickey and the gang as they duck the halls!",
"type": "Movie",
"title": "Duck the Halls: A Mickey Mouse Christmas Special"
}
},
{
"_index": "disney",
"_id": "5l_eAYMB9kKbW3vlFVrc",
"_score": 1,
"_source": {
"duration": "91 min",
"listed_in": "Comedy",
"cast": "Jim Varney, Noelle Parker, Douglas Seale",
"date_added": "November 26, 2021",
"show_id": "s2",
"director": "John Cherry",
"release_year": 1988,
"rating": "PG",
"description": "Santa Claus passes his magic bag to a new St. Nic.",
"type": "Movie",
"title": "Ernest Saves Christmas"
}
},
{
"_index": "disney",
"_id": "51_eAYMB9kKbW3vlFVrc",
"_score": 1,
"_source": {
"country": "United States",
"show_id": "s3",
"director": "Karen Disher",
"release_year": 2011,
"rating": "TV-G",
"description": "Sid the Sloth is on Santa's naughty list.",
"type": "Movie",
"title": "Ice Age: A Mammoth Christmas",
"duration": "23 min",
"listed_in": "Animation, Comedy, Family",
"cast": "Raymond Albert Romano, John Leguizamo, Denis Leary, Queen Latifah",
"date_added": "November 26, 2021"
}
--------------OUTPUT TRUNCATED-----------------------------------
Using Elasticsearch Scan Search Type
Another method of fetching all the records from an index is using the scan search type. The syntax is as shown:
{
"query": {
"match_all": {}
}
}
It is good to remember that the scan search is deprecated in recent versions of Elasticsearch. It is recommended to use the scroll feature. You can learn more about that in the scroll elastic search tutorial.
An example is as shown:
{
"query": {
"match_all": {}
}
}'
Similarly, the request above should fetch all the documents in the index as shown in the sample output below:
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1450,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "disney",
"_id": "5V_eAYMB9kKbW3vlFVrc",
"_score": 1,
"_source": {
"duration": "23 min",
"listed_in": "Animation, Family",
"cast": "Chris Diamantopoulos, Tony Anselmo, Tress MacNeille, Bill Farmer, Russi Taylor, Corey Burton",
"date_added": "November 26, 2021",
"show_id": "s1",
"director": "Alonso Ramirez Ramos, Dave Wasson",
"release_year": 2016,
"rating": "TV-G",
"description": "Join Mickey and the gang as they duck the halls!",
"type": "Movie",
"title": "Duck the Halls: A Mickey Mouse Christmas Special"
}
},
---------------------OUTPUT TRUNCATED--------------------
Conclusion
In this post, you discovered two main methods of fetching all the documents from a given Elasticsearch index.
