User data encryption promotion

Promotion Steps:

• Promotion of encryption service

  • Choose values for following fields

    • master-password: choose any string of any length (can contain alphanumerics and special characters)

    • master-salt: choose any string of length 8 (can contain alphanumerics and special characters)

    • master-initialvector: choose any string of length 12 (can contain alphanumerics and special characters)

    • Ask Devops to generate keys for above selected values,

    • In environment secrets.yml file, add “egov-enc-service” subsection under “secrets” section and provide values for above three fields. For ex:- for Dev environment https://github.com/egovernments/eGov-infraOps/blob/master/helm/environments/dev-secrets.yaml#L29 (Ask Devops to do it)

  • add field “state-level-tenant-id“ under “egov-enc-service:” section for state level tenantId in environment yml. ex:- https://github.com/egovernments/eGov-infraOps/blob/master/helm/environments/qa.yaml#L525

  • Promote egov-enc-service:4-master-f47bff2

  • Make sure “egov-enc-service“ entry is present in “egov-service-host” in environment yml ,ex:- for dev https://github.com/egovernments/eGov-infraOps/blob/master/helm/environments/dev.yaml#L65 . If not, make changes and build and deploy zuul from master branch.

• Data migration steps (migration script and config in attachment):-

  • Provide DB details in following environment variables

    • DB_PASSWORD

    • DB_HOST

    • DB_PORT

    • DB_USERNAME

    • DB_NAME'

  • Backup old tables

    • create table eg_user_backup_plaintext as (select * from eg_user);

    • create table eg_user_address_backup_plaintext as (select * from eg_user_address);

  • Delete foreign key referenced on ‘eg_user’ from ‘eg_userrole_v1’ temporarily until the data is transformed

    • ALTER TABLE eg_userrole_v1 DROP CONSTRAINT fk_user_role_v1;

  • Deploy user service build with encryption to run flyway migration (egov-user:11-user_changes_MT-800f319)

  • Clean tables of all plain text data

    • delete from eg_user_address;

    • delete from eg_user;

  • run migration

    • Script python package dependencies

      • import psycopg2

      • import sys

      • import json

      • import requests

      • import configparser

      • import logging

      • import os

    • Commands to run for migration

      • python3 user_migration.py config_user_encryption.txt

      • python3 user_migration.py config_address_encryption.txt

  • Restore earlier deleted foreign key constraint

    • ALTER TABLE eg_userrole_v1 ADD CONSTRAINT fk_user_role_v1 FOREIGN KEY (user_id, user_tenantid) REFERENCES eg_user(id, tenantid) MATCH SIMPLE ON UPDATE NO ACTION ON DELETE NO ACTION;
      
      

• Service Builds:-

  • User service:-  egov-user:11-user_changes_MT-800f319

    • Set environment variable “DECRYPTION_ABAC_ENABLED” to false

  • User service copy for chatbot:- egov-user-chatbot:4-user_changes_MT-621fe60

• Report service:- report:22-report-encryption-changes-e92c8ae

• enc-service:- egov-enc-service:4-master-f47bff2

• MDMS:

  • Copy mdms folder https://github.com/egovernments/egov-mdms-data/tree/master/data/pb/DataSecurity

Open

Impact analysis:

The following columns in following tables are encrypted as part of user data  encryption

• User service api response: The apis of user service after user data encryption are unchanged (given accessed using suitable roles), services using user service apis will not be affected. Please note if any user who does not have any role defined in “DecryptionABAC” (https://github.com/egovernments/egov-mdms-data/blob/master/data/pb/DataSecurity/DecryptionABAC.json ) master’s “ALL_ACCESS” section, he will receive encrypted data on calling user service apis. To get decrypted data the user should have at least one role defined in  “ALL_ACCESS” section. If he contains a matching role the user service response will be as earlier. For providing full decryption to any role on accessing user service apis add role entry with all fields names to be decrypted.

• Searcher: -  Since the data in the user tables is encrypted, no service should directly pick data from DB. Services using searcher should pick user data from user service separately which will provide decrypted data or searcher should be enhanced to enrich user data by calling user service

• Dashboard:- No dashboard is based on user PIIs,

  • In PT,TL,PGR Currently user PIIs data is going on tl-index, bpa-index, paymentsindex-v1,bpastakeholderindex, pgrindex, ptindex-v1 indexes
    
    

• Reports: - 

  • By default any report picking data directly from user tables will have encrypted values

  • Reports can use RBAC supported by encryption service for enccrypted column decryption

  • To provide access to any role in any report to decrypt encrypted user data columns, please use following example to add entry in DecryptionABAC master (https://github.com/egovernments/egov-mdms-data/blob/master/data/pb/DataSecurity/DecryptionABAC.json ) and adding key in report config  

    • Ex:-   In “TradeLicenseRegistryReport” report, To provide “EMPLOYEE” role “name”,”mobilenumber” columns (please note column name should match with column names defined in the report config) decryption access we made two changes

      • Added   “decryptionPathId: TradeLicenseRegistryReport” key value in report config of “TradeLicenseRegistryReport"(https://github.com/egovernments/configs/blob/master/reports/config/rainmaker-tl-reports.yml )

      • And added following entry in DecryptionABAC master (https://github.com/egovernments/egov-mdms-data/blob/master/data/pb/DataSecurity/DecryptionABAC.json )