Detailed Steps to configure privacy in W&S module

Overview:
To make sure that after enabling privacy, the system works as expected, we will require some configurations to be made in the environment. This document contains all the steps to ensure successful implementation and working of Water & Sewerage module.

Steps:

Following will be the changes required to move it to other environments:

1. Add a new role for REINDEXING so as to push encrypted data in the index in roleactions.json and roles.json files.
   Reference for these file changes can be taken from the following commit: https://github.com/egovernments/egov-mdms-data/pull/2835/files (Please pick the changes for the required files- roleactions.json and roles.json only)
   
   

2. Add a copy of the existing water-service index with a different topic name(update-ws-encryption and update-sw-encryption) for the encryption process.
   Reference: https://github.com/egovernments/configs/pull/2510/commits/bf025f1489933dce71fe448d1181515fa418f09c

    

3. Some changes need to be made for existing indexes in water-service and sewerage-service indexer files. The changes with respect to these files can be referred from the following commit:
   https://github.com/egovernments/configs/pull/2342
   https://github.com/egovernments/configs/pull/2345
   
   

4. Restart the indexer.
   
   

5. Add the following json mappings in the existing mappings (parallely to water-services and sewerage-services key) for water-services and sewerage-services in kibana so that the PII data is not visible during search(The data do remain in the index and also search with respect to this happens as is).
   
   

   "_source": { "excludes": [ "Data.ownerMobileNumbers", "Data.connectionHolders.ownerType", "Data.connectionHolders.gender", "Data.connectionHolders.mobileNumber", "Data.connectionHolders.correspondenceAddress", "Data.connectionHolders.fatherOrHusbandName", "Data.connectionHolders.relationship", "Data.plumberInfo.mobileNumber" ] }

   Sample index at bottom.
   
   

6. Add 2 new persister files responsible for managing old data encryption. ws-enc-audit-persister.yml and sw-enc-audit-persister.yml.
   
   

7. Update the path of these files in the DevOps repo in the specific environment file.
   
   

8. Restart the persister
   
   

9. Deploy new ws-service and sw-service builds.
   
   

10. Add the following file in the indexer: privacy-audit.yaml
    
    

11. Update path for the above index in DevOps environment file: file:///work-dir/configs/egov-indexer/privacy-audit.yaml here.
    
    

12. Add searchpath for ws and sw inbox :
    "NewWS1,ModifyWSConnection,DisconnectWSConnection":{"searchPath":"http://ws-services.egov:8080/ws-services/wc/_search","dataRoot":"WaterConnection","applNosParam":"applicationNumber","businessIdProperty":"applicationNo","applsStatusParam":"applicationStatus"},"NewSW1,ModifySWConnection,DisconnectSWConnection":{"searchPath":"http://sw-services.egov:8080/sw-services/swc/_search","dataRoot":"SewerageConnections","applNosParam":"applicationNumber","businessIdProperty":"applicationNo","applsStatusParam":"applicationStatus"}"NewWS1,ModifyWSConnection,DisconnectWSConnection":{"searchPath":"http://ws-services.egov:8080/ws-services/wc/_search","dataRoot":"WaterConnection","applNosParam":"applicationNumber","businessIdProperty":"applicationNo","applsStatusParam":"applicationStatus"},"NewSW1,ModifySWConnection,DisconnectSWConnection":{"searchPath":"http://sw-services.egov:8080/sw-services/swc/_search","dataRoot":"SewerageConnections","applNosParam":"applicationNumber","businessIdProperty":"applicationNo","applsStatusParam":"applicationStatus"}here
    
    

13. Following variables need to be added in env-secrets file :

    elasticsearch: egov-es-username egov-es-password

     

14. Following variables need to be overridden in env file:

    #W&S indexes water.es.index=water-services sewerage.es.index=sewerage-services

     

15. Port-forward the ws-service and sw-service pods and hit the curl to start encryption.
    The curls can be referred from here:
    Water-encryption curl:
    
    

    
    Sewerage-encryption curl:
    
    

    
    In the params list in both the above curls, “tenantIds” param can either be provided with a single tenantId or a list of tenantIds for encrypting the data with respect to the provided tenantIds. However, to encrypt the data for all tenantIds in the system, tenantIds param itself should be removed.
    
    

To validate if the encryption is completed, you can check with the following dB queries:

• select * from eg_ws_enc_audit order by createdtime desc;

• select count(*) from eg_ws_id_enc_audit;

• select * from eg_ws_id_enc_audit;

Sample Index for point 5: