Automatic shard rebalancing conforms to restrictions and rules like allocation filtering and forced awareness, leading to the most efficient and well-balanced cluster possible.
NOTE: Do not confuse shard reallocation, which is the process of finding and moving unassigned shards to the nodes in which they reside, with rebalancing. Rebalancing takes assigned shards and moves them evenly to various nodes, the purpose being the equal distribution of shards per node.
How to Enable Automatic Rebalancing
To enable automatic cluster rebalancing in Elasticsearch, we can use the PUT request to_cluster API endpoint and add the settings we need.
The settings available for dynamic shard rebalancing include:
- cluster.routing.rebalance.enable: Controls automatic rebalancing for various shard types, such as:
- All: Sets enable shard rebalancing for all indices.
- None: Disables shard rebalance for all indices.
- Replicas: Only replica shard rebalance is allowed.
- Primary: Only primary shard rebalancing is allowed.
- cluster.routing.allocation.allow_rebalance: Sets the value for shard rebalancing. Options include:
- Always: Enables rebalancing indefinitely.
- Indices_primaries_active: Allows rebalancing only when all primary shards in the cluster are allocated.
- Indices_all_active: Allows rebalancing when only the shards in the cluster are allocated. This includes both the primary and the replica shards.
- cluster.routing.allocation.cluster.concurrent.rebalance: This option sets the number of concurrent rebalances allowed in the cluster. The default value is 2.
Consider the request below to allow automatic shard rebalancing for the cluster.
{
"persistent": {
"cluster.routing.rebalance.enable": "primaries",
"cluster.routing.allocation.allow_rebalance": "always" ,
"cluster.routing.allocation.cluster_concurrent_rebalance":"2"
}
}
The following is the cURL command:
This command should return a response as the JSON object acknowledges the settings that are updated.
“acknowledged”: true,
"persistent" : {
"cluster" : {
"routing" : {
"rebalance" : {
"enable" : "primaries"
},
"allocation" : {
"allow_rebalance" : "always",
"cluster_concurrent_rebalance" : "2"
}
}
}
},
"transient" : { }
}
Manual Index Rebalancing
You can also rebalance a shard manually for a specific index. I would not recommend this option because the Elasticsearch default rebalancing options are very efficient.
However, should the need to perform manual rebalancing arise, you can use the following request:
“acknowledged”: true,
"persistent" : {
"cluster" : {
"routing" : {
"rebalance" : {
"enable" : "primaries"
},
"allocation" : {
"allow_rebalance" : "always",
"cluster_concurrent_rebalance" : "2"
}
}
}
},
"transient" : { }
}
The cURL command is:
NOTE: Keep in mind that if you perform a manual rebalance, Elasticsearch may move the shards automatically to ensure the best rebalance possible.
Conclusion
This guide walked you through updating and modifying the settings for an Elasticsearch cluster to enable automatic shard rebalancing. The article also covered manual rebalancing, if you require it.
