DSS-COLLECTION

Step-1: Delete the index for dss-collection

curl -XDELETE "http://elasticsearch-client-v1.es-cluster:9200/egov-dss-ingest-enriched"
curl -XDELETE http://elasticsearch-client-v1.es-cluster:9200/dss-collection_v1

Step-2: Create new index

curl -XPUT -H "Content-Type: application/json"  http://elasticsearch-client-v1.es-cluster:9200/egov-dss-ingest-enriched?pretty

Step-3: Create a new alias

curl -XPOST "http://elasticsearch-client-v1.es-cluster:9200/_aliases" -H 'Content-Type: application/json' -d'
{
   "actions" : [
       { "add" : { "index" : "egov-dss-ingest-enriched", "alias" : "dss-collection_v1" } }
   ]
}'

Step-4: Delete the Kafka connector and create a new one

curl -X DELETE http://kafka-connect.kafka-cluster:8083/connectors/cms-case-es-sink9132

Step-5: Create a new Kafka connector

curl -X POST \
http://kafka-connect.kafka-cluster:8083/connectors/ \
-H 'Content-Type: application/json' \
-H 'Cookie: SESSIONID=f1349448-761e-4ebc-a8bb-f6799e756185' \
-H 'Postman-Token: adabf0e8-0599-4ac9-a591-920586ff4d50' \
-H 'cache-control: no-cache' \
-d '{
"name": "cms-case-es-sink9132",
"config": {
"connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector",
"connection.url": "http://elasticsearch-client-v1.es-cluster:9200",
"type.name": "general",
"topics": "egov-dss-ingest-enriched",
"key.ignore": "false",
"schema.ignore": true,
"value.converter.schemas.enable": false,
"key.converter": "org.apache.kafka.connect.storage.StringConverter",
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"transforms": "TopicNameRouter",
"transforms.TopicNameRouter.type": "org.apache.kafka.connect.transforms.RegexRouter",
"transforms.TopicNameRouter.regex": ".*",
"transforms.TopicNameRouter.replacement": "dss-collection_v1",
"batch.size": 10,
"max.buffered.records": 500,
"flush.timeout.ms": 600000,
"retry.backoff.ms": 5000,
"read.timout.ms": 10000,
"linger.ms": 100,
"max.in.flight.requests": 2,
"errors.log.enable": true,
"errors.deadletterqueue.topic.name": "dss-collection_v1-es-failed",
"tasks.max": 1
}
}'

Step-6: Once Kafka connector is created, forward port 8080 for index pod

kubectl port-forward <indexer_name> -n egov 8080:8080

Step-7: Now run the reindexing curl command from outside the playground pod

curl --location --request POST 'http://localhost:8080/egov-indexer/index-operations/_legacyindex' \
--header 'Cache-Control: no-cache' \
--header 'Content-Type: application/json' \
--header 'Postman-Token: d380bebb-383f-1b7c-76d1-10c1dc07dd06' \
--data-raw '{
  "RequestInfo": {
    "apiId": "string",
    "ver": "string",
    "ts": null,
    "action": "string",
    "did": "string",
    "key": "string",
    "msgId": "string",
    "authToken": "b843ef27-1ac6-49b8-ab71-cd0c22f4e50e",
    "correlationId": "e721639b-c095-40b3-86e2-acecb2cb6efb",
    "userInfo": {
      "id": 23299,
      "uuid": "e721639b-c095-40b3-86e2-acecb2cb6efb",
      "userName": "9337682030",
      "name": "Abhilash Seth",
      "type": "EMPLOYEE",
      "mobileNumber": "9337682030",
      "emailId": "abhilash.seth@gmail.com",
      "roles": [
	{
	  "id": 281,
	  "name": "Employee"
	}
      ]
    }
  },
  "apiDetails": {
    "uri": "http://collection-services:8080/collection-services/payments/_search",
    "tenantIdForOpenSearch": "uk.dehradun",
   
    "paginationDetails": {
      "offsetKey": "offset",
      "sizeKey": "limit",
      "maxPageSize": 100
    },
    "responseJsonPath": "$.Payments"
  },
  "legacyIndexTopic": "egov-payment-legacy-index",
  "tenantId": "uk.dehradun"
}'

Egov-Impel

DSS-COLLECTION

Related content