Ifix-Data-Migration

The functionality of this task is to to migrate the Master data and the fiscal event from Mongo-db to Postgres

STEP 1: EXISTING DATA MIGRATION FROM MONGO TO IFIX
We have to migrate the following collection from mongo to postgres.

From MGRAMSEVA NAMESPACE MONGO.

COLLECTION NAME IN MGRAMSEVA MONGO	TABLE NAME IN MGRAMSEVA POSTGRES

COLLECTION NAME IN MGRAMSEVA MONGO	TABLE NAME IN MGRAMSEVA POSTGRES
department	department
expenditure	expenditure
project	project
departmentEntity	department_entity
departmentHierarchyLevel	department_hierarchy_level
departmentEntity(children details)	department_entity_children (this have mapping of parent-child of department entity)

FROM IFIX NAMESPAECE MONGO

COLLECTION NAME IN IFIX MONGO	TABLE NAME IN IFIX POSTGRES

COLLECTION NAME IN IFIX MONGO	TABLE NAME IN IFIX POSTGRES
chartOfAccount	eg_ifix_chartofaccount
fiscal_event	eg_ifix_fiscal_event
fiscal_event(amount details)	eg_ifix_amount_detail
fiscal_event(reciever details)	eg_ifix_receivers

1. We have two fix-migration-toolkit-service running inn IFIX and Mgramseva
ifix-migration-toolkit : build “ifix-migration-toolkit:develop-2557812-9 “ in mgramseva namespace
ifix-migration-toolkit : build “egovio/ifix-migration-toolkit:develop-b7a2050-13” in Ifix namespace
Git repo: https://github.com/egovernments/iFix-Dev/tree/develop/utilities/ifix-migration-toolkit
The functionality of this service is to call the plain search api of concerned service that connnects to mongo db of those service and fetch the result and migrate those to postgres db. We have four APIs to migrate data from mongo to postgres
https://api.postman.com/collections/24408626-b519a61e-034c-4f3f-b4ee-7829dc9e1b67?access_key=PMAT-01H8H1MB52HJ0FY3MQQ9FXA73E

2. Services and branch details you need to deploy to get the db details from Mongo db.
a. Ifix-Department-entity-service in mgramseva namespace:
Branch and git repo: https://github.com/egovernments/iFix-Dev/tree/mongodb-plainsearch/reference-adapter/ifix-department-entity-service
b. Adapter-master-data-service in mgramseva namespace:
Branch and git repo: https://github.com/egovernments/iFix-Dev/tree/mongodb-plainsearch/reference-adapter/adapter-master-data-service

c. Ifix-master-data-service in Ifix namespace:
Branch and git repo: https://github.com/egovernments/iFix-Dev/tree/mongodb-plainsearch/domain-services/ifix-master-data-service
d. Fiscal-event-service in Ifix namespace:
Branch and git repo: https://github.com/egovernments/iFix-Dev/tree/mongodb-plainsearch/domain-services/fiscal-event-service

Build and deploy the branch “mongodb-plansearch” for these above services
Before deploying add the perisiter and the devopps file changes mentioned in step 4 and step 6

3. Once done check the count of records in mongo-db of both mgramseva and Ifix namespace and run the apis shared in point 1
After successfully running the apis check for the counts in postgres and compare the counts.

4. Persister details:
Adapter-master-data: https://github.com/misdwss/config-mgramseva/blob/UAT/egov-persister/adapter-master-data-service.yml
ifix-department-entity: https://github.com/misdwss/config-mgramseva/blob/UAT/egov-persister/department-entity-persister.yml
Ifix-master-data-service: https://github.com/misdwss/config-mgramseva/blob/master/egov-persister/ifix-master-data-persister.yml
fiscal-evenn-service: https://github.com/misdwss/config-mgramseva/blob/master/egov-persister/fiscal-events-persister.yml

5. API curls.
To Migrate Department: portforward to adapter-master-service

curl --location 'http://localhost:8080//ifix-migration-toolkit/project/v1/_migrate' \
--header 'authority: mgramseva-qa.egov.org.in' \
--header 'accept: */*' \
--header 'accept-language: en-US,en;q=0.9' \
--header 'content-type: application/json; charset=utf-8' \
--header 'cookie: _ga=GA1.1.70463156.1677473784; _ga_8H6W5DYGX0=GS1.1.1677557215.2.1.1677557273.0.0.0' \
--header 'origin: https://mgramseva-qa.egov.org.in' \
--header 'referer: https://mgramseva-qa.egov.org.in/mgramseva/home/consumercreate' \
--header 'sec-ch-ua: "Chromium";v="110", "Not A(Brand";v="24", "Google Chrome";v="110"' \
--header 'sec-ch-ua-mobile: ?0' \
--header 'sec-ch-ua-platform: "macOS"' \
--header 'sec-fetch-dest: empty' \
--header 'sec-fetch-mode: cors' \
--header 'sec-fetch-site: same-origin' \
--header 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36' \
--data '{
    "requestHeader": {
        "apiId": "mgramseva",
        "ver": 1,
        "ts": "",
        "action": "POST",
        "did": 1,
        "key": "",
        "msgId": "",
        "authToken": "56d054b3-206e-4a25-8c67-cbef9aa59723",
        "userInfo": null
    },
   "tenantId":"pb"

}'

To migrate Expenditure

curl --location 'http://localhost:8080//ifix-migration-toolkit/expenditure/v1/_migrate' \
--header 'authority: mgramseva-qa.egov.org.in' \
--header 'accept: */*' \
--header 'accept-language: en-US,en;q=0.9' \
--header 'content-type: application/json; charset=utf-8' \
--header 'cookie: _ga=GA1.1.70463156.1677473784; _ga_8H6W5DYGX0=GS1.1.1677557215.2.1.1677557273.0.0.0' \
--header 'origin: https://mgramseva-qa.egov.org.in' \
--header 'referer: https://mgramseva-qa.egov.org.in/mgramseva/home/consumercreate' \
--header 'sec-ch-ua: "Chromium";v="110", "Not A(Brand";v="24", "Google Chrome";v="110"' \
--header 'sec-ch-ua-mobile: ?0' \
--header 'sec-ch-ua-platform: "macOS"' \
--header 'sec-fetch-dest: empty' \
--header 'sec-fetch-mode: cors' \
--header 'sec-fetch-site: same-origin' \
--header 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36' \
--data '{
    "requestHeader": {
        "apiId": "mgramseva",
        "ver": 1,
        "ts": "",
        "action": "POST",
        "did": 1,
        "key": "",
        "msgId": "",
        "authToken": "56d054b3-206e-4a25-8c67-cbef9aa59723",
        "userInfo": null
    },
   "tenantId":"pb"

}'

To migrate department-hierarchy port forward to ifix-department-entity-service

curl --location 'http://localhost:8080//ifix-migration-toolkit/department_hierarchy/v1/_migrate' \
--header 'authority: mgramseva-qa.egov.org.in' \
--header 'accept: */*' \
--header 'accept-language: en-US,en;q=0.9' \
--header 'content-type: application/json; charset=utf-8' \
--header 'cookie: _ga=GA1.1.70463156.1677473784; _ga_8H6W5DYGX0=GS1.1.1677557215.2.1.1677557273.0.0.0' \
--header 'origin: https://mgramseva-qa.egov.org.in' \
--header 'referer: https://mgramseva-qa.egov.org.in/mgramseva/home/consumercreate' \
--header 'sec-ch-ua: "Chromium";v="110", "Not A(Brand";v="24", "Google Chrome";v="110"' \
--header 'sec-ch-ua-mobile: ?0' \
--header 'sec-ch-ua-platform: "macOS"' \
--header 'sec-fetch-dest: empty' \
--header 'sec-fetch-mode: cors' \
--header 'sec-fetch-site: same-origin' \
--header 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36' \
--data '{
    "requestHeader": {
        "apiId": "mgramseva",
        "ver": 1,
        "ts": "",
        "action": "POST",
        "did": 1,
        "key": "",
        "msgId": "",
        "authToken": "56d054b3-206e-4a25-8c67-cbef9aa59723",
        "userInfo": null
    },
   "tenantId":"pb",
   "postgresToES": true

}'

To migrate Department entity

There are Intermediate calls that are made to get the details from ifix
for example when you migrate fiscal event it will call this migrate api

which in turns call this curl to get the details from fiscal-event-service

DevOps file change :
https://github.com/misdwss/iFix-DevOps/pull/841/files
https://github.com/misdwss/iFix-DevOps/pull/842/files?diff=split&w=0
https://github.com/misdwss/iFix-DevOps/pull/843/files
https://github.com/misdwss/iFix-DevOps/pull/844/files
https://github.com/misdwss/iFix-DevOps/pull/846/files