Prequisites : Elasticdump
The indexes available in an Elasticsearch environment can be known as follows :
curl http://es-host:9200/_cat/indices?v
The indexes can be copied from one environment (say Production) to another (say QA) as follows :
elasticdump --input=http://production-es-host:9200/index --output=http://qa-es-host:9200/index
For this to work, the QA ES endpoint should be accessible from the Prod environment or vice-versa.
If, for some reason, this is not met (say Production and QA are two different AWS accounts and VPC peering is ruled out), the Prod ES indexes can be dumped to a JSON file which can be then be restored in the QA environment.
elasticdump --input=http://production-es-host:9200/my_index --output=prod_index.json
The same can be restored in the QA environment as follows :
elasticdump --input= prod_index.json --output=http://qa-es-host:9200/my_index
Sometimes, the following error is thrown when indexes are getting restored.
error: {
type: 'cluster_block_exception',
reason: 'blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];'
}
This occurs because in its default configuration, Elasticsearch will not allocate any more disk space when more than 90% of the disk is used overall. (i.e. by Elasticsearch or other applications). This watermark can be set lower but this may prevent important applications from being able to properly allocate disk space.
A way out is to increase the size of the destination ES cluster (according to the size of the source cluster).
The capacity of the ES cluster (for the source/destination end) can be checked as follows :
curl -XGET 'http://es-host:9200/_cat/allocation?v'
If, for example, the elasticsearch-data uses a PersistentVolumeClaim, the same can be edited to increase the size using edit pvc pvcname. This capacity can only be increased if the underlying storage class has AllowVolumeExpansion set to true.