“Memory is an essential but limited resource when working with Elasticsearch. This is because Lucene will make use of every available memory. However, when misconfigured, the memory settings can lead to low performance and inefficient memory use.”
In this tutorial, we will show you the maximum and minimum JVM Heap Size configuration when working with Elasticsearch.
Let’s get started.
What is a Heap Memory?
In the context of Elasticsearch, Heap memory refers to the total amount of memory allocated to the Java Virtual Machine within an Elasticsearch node.
Elasticsearch will default set the JVM Heap size based on the total memory of the base system and the node’s role. This means the heap memory size allocation may vary depending on whether it’s a master node, data, ingest, data_cold, etc.
For most production environments, allowing Elasticsearch to manage the heap size is recommended and more than sufficient.
NOTE: If you are running Elasticsearch in Docker, the total heap memory is based on the total size of the docker container and not the Docker’s host.
Configuring Minimum and Maximum Heap Size
To configure the minimum and maximum heap size, we can use the Xms and Xmx parameters. Elasticsearch probits are setting the maximum memory to no more than 50% of the total memory. This is because apart from JVM Heap, Elasticsearch requires more memory for other operations such as filesystem cache, network communication, etc. Similarly, the JVM will use a section of the remaining 50% memory.
Second, do not set the xms and xmx values to more than the threshold of the oops. For safe configuration, limit it to 26GB or 30GB on some systems.
You can check the threshold in the Elasticsearch log.
You should see an entry as shown:
[2022-08-19T20:08:07,207][INFO ][o.e.e.NodeEnvironment ] [debian11] heap size [1.9gb], compressed ordinary object pointers [true]
[2022-08-19T20:36:47,244][INFO ][o.e.e.NodeEnvironment ] [debian11] heap size [1.9gb], compressed ordinary object pointers [true]
You can also query the nodes information API for the xms and xmx values:
You should see an output as shown:
Set the Minumum and Maximum Heap Size
To modify the values of the JVM Heap Size, you need to add a configuration file in the /etc/elasticsearch/jvm.options.d directory. This file should end with the .options extension.
For example:
Edit the file
Add the desired minimum and maximum heap memory size.
For example, the entry below configures the minimum and maximum heap size to 4GB.
Save the file and restart the Elasticsearch service.
Conclusion
In this tutorial, you learned the JVM Heap in the context of Elasticsearch, how Elasticsearch configures the JVM Heap, and how you can modify the heap size.
Thanks for reading!!
