Purpose

Request: To take the Punjab W&S ERP elasticsearch index and restore it on Punjab Rainmaker elasticsearch cluster.

Environment: Punjab W&S ERP elasticsearch (Production), Punjab Rainmaker elasticsearch cluster (UAT)

Elasticsearch Indecies needs to dump:
1. receipts-consumers
2. demandbillconsumers
3. waterchargesconsumers
4. projecteddemand-consumers
5. applications-receipts
6. pwssbulbs
7. seweragechargesconsumers
8. wsapplications
9. nonpwssbulbs
10. dwssulbs

Solution Approach

Procedure:

1. From Punjab W&S ERP elasticsearch cluster, taken snaphots of 10 indecies using curl.
2. Pushed taken snapshots from the server to egov-aws s3 bucket.

Approach-1:

1. Trying to restore the snapshot from s3 bucket to Punjab Rainmaker elasticsearch cluster (UAT).
2. First locally by setting up elasticsearch cluster using minikube, here we are trying to push the elasticsearch snaphots taken to the es cluster.
3. Rainmaker UAT es cluster has, client, data, and master nodes.
4. Using helm elasticsearch chart, setup the elasticsearch cluster with client, data, and master node in minikube
5. Take the snaphot from the server to local
6. Set the snapshot repo path which is the snaphot local path
7. Connect to the kubernetes client and execute to get into.
8. Using kubectl cp command try to push the local elasticsearch snaphot to the elasticsearch client
9. It is observed that client, data, and master nodes were rejecting the push
10. By this realized that es cluster (master,data,client) doesn't have a shared resource or filesystem to where we are trying push the snaphot from locally

Approach-2:

1. Decided to use elasticdump an npm tool this will help in doing index to index snaphot dumping
2. now using elasticdump command we restored the ERP elasticsearch index to elasticsearch cluster which is in mcroserivces.

elasticdump --input=http://<erp-elasticsearch-endpoint>/<index> --output=http://<rm-elasticsearch-client-endpoint>:9200/<index>

http://<erp-elasticsearch-endpoint>/<index>: erp es doesnt have endpoint exposed, hence it is using localhost

http://<rm-elasticsearch-client-endpoint>:9200/<index>: http://elasticsearch-client-v1.es-cluster:9200/

Getting ERP elasticsearch endpoint:

1. Punjab ERP production elasticsearch is hosted, and configured using nginx.
2. Here in nginx using virtual host configuration at the elasticsearch we have to create end point
3. Using nginx config file modify at the

4. vi /etc/nginx/sites-enabled/default
   server {
           listen 81 default_server;
           listen [::]:81 default_server;
   
           server_name <serverIP>;
   
   	location / {
               proxy_pass http://localhost:9200;
               proxy_http_version 1.1;
               proxy_set_header Upgrade $http_upgrade;
               proxy_set_header Connection 'upgrade';
               proxy_set_header Host $host;
               proxy_cache_bypass $http_upgrade;
           }
   
       }
   save & exit

5. /etc/init.d/nginx reload

6. Open the 81 port at the security group setting of the server level to 0.0.0.0/0

7. so we got our ERP elasticsearch endpoint: http://<elasticsearch-server-ip>/<index>

Now we will  do index to index elasticdump:

1. Using elasticdump it is a npm based tool so we need to install npm elasticdump tool in es-client to perfrom the elasticdumps index to index which is not recommended.
2. So it is observered and by guess found that playground service in Punjab rainmaker present within the UAT cluster, fortunately elasticdump tool was already installed on it.
3. using playground service, we dive into and using elasticdump command we restored the ERP elasticsearch index to elasticsearch cluster index

Conclusion 

From this using elasticdump command we are able to restore the ERP elasticsearch index to RM index. With total of 10 indices, and total size of 4.5 gb