Demo Instance Set up and Upgrade Process

Objective

This document aims at defining the process to be followed for the set-up and management of the demo environment. For the Urban mission, a demo instance is already in place and this has to be upgraded after every major release of DIGIT.

Key stakeholders

DIGIT demo instance is targeted to be used by the following people-

eGov team
Implementation partners
Ecosystem partners
Other people accessing the digit website, who are interested in DIGIT

Standard operating procedures

There needs to be an infrastructure provisioned in the cloud for setting up the demo environment. This will have the minimum configurations to start with and it can be scaled up based on the volume of usage. This instance is to be up and running 24 hours on all days (exception only when the upgrade happens). SMS server to be configured with sufficient credits so that OTP will be made available for citizen login.

Activity	Description of Activity	Frequency	Dependency	Responsibility	Duration	Timeline	Publish
Decide on the downtime for upgrade	After mutual agreement with the external and internal stakeholders decide on the date for upgrade	One time	Release communication	RP from Tech team and Partnership team	3-5 days	T	Email confirmation from partnership team for go ahead
Take Back up	Take backup of DB and images	One time	Get confirmation on upgrade downtime	RP from Tech team	2 hrs	T+1
Send Communication to partners on downtime	Emails to folks who downloaded credentials for instance	One time	Get confirmation on upgrade downtime	RP from Partnership team	1 hr	T+1	Emails sent out to partners
Update DIGIT website	Access DIGIT page to be updated on the downtime	One time	Get confirmation on upgrade downtime	RP from Documentation team	2 hrs	T+1	Website updated
Deploying new builds and configurations	Based on the release notes update the builds and configurations	Multiple times	After backup	RP from Tech team	2 days	T+3	All configurations and build available on the environment
Loading localisation	Load the english and hindi localisation as per the release notes	One time	New builds deployed	QA	1 day	T+4	All localisation keys available on the environment
Testing	Sanity testing for all the applications on City A and City B	Multiple times	New builds deployed	Tech team and QA	1-2 days	T+6	Application working as expected
QA sign off	QA does final sanity	One time	Unit testing completed	QA	2 day	T+8	Communication to product team for product sign-off
Product sign off	Product team to do sanity testing of all modules with City A and City B	One time	QA sign off given	Product Owner	2-4 days	T+12	Communication to RP from Tech team
Communication of upgrade completion	Mail to be send to all stakeholders about the upgrade completion	One time	Product sign off	RP from Tech team	2 hrs	T+13	Email communication
Send Communication to partners on upgrade completion	Emails to folks who downloaded credentials for instance	One time	Validate access digit link. Upgrade completion email from Tech team	RP from Partnership team	1 hr	T+13	Emails sent out to partners
Update DIGIT website	Remove from the DIGIT website about the downtime banner	One time	Communication on upgrade completion	RP from Documentation team	2 hr	T+13	Website retained back to how it was prior to upgrade

When to upgrade

The upgrade should be done ideally after one week of release and before the release webinar. Normally DIGIT will have one release per quarter; this way the upgrade will be a once in a quarter affair. However, if there are critical patch releases done, the same needs to be replicated in the demo environment too.

Ownership

Below listed are the teams involved in upgrading and maintaining the demo instance -

Product Team - They are the primary owner of this instance. They will provide the master data to be loaded into this instance. Localization keys, SMS templates and workflow configurations are to be shared by the product owners.
Technical Team (Engineering and Implementation) - Set-up, configuration and ongoing upgrades are done by this team. To start with one person from the engineering and implementation team. These two will be coordinating all the communication related to the upgrades with other teams.
QA Team - One QA engineer will be aligned for this environment qualification.
DevOps Team - Any infrastructure and secret key related tasks for the set-up and upgrade will be done by this team.
Partnership Team - All communications to the external stakeholders like notifying the demo instance downtime and upgrade completed are done by this team.
Documentation Team - This team will help in updating the website with the downtime

Communication process

Once the engineering team announces the DIGIT release officially, the partnership team is informed about the tentative dates for the demo environment upgrade. They will communicate the same with the partners and get a formal buy-in from them. In case the partners have some critical demos to be done on the proposed dates, eGov is open to pushing the upgrade for a couple of days.

Once the upgrade dates are agreed upon, the digit website will be updated with the downtime information. Generally, it will take one week to upgrade and certify the same. All the teams involved in this process are informed about this activity upfront, for better planning.

How to upgrade

The demo environment is configured for two tenants majorly- City A and City B. Owners from the engineering team and implementation team will do the following-

Follow the respective release notes published on the website and make all changes mentioned.
All master data and configurations will be done as per the instruction of the product team.
Before deploying the new builds take a snapshot of the existing images and take a backup of the database.
If any new SMS are introduced, then template to be received from the SMS provider and append with localization keys.
If any secret keys are required as part of the upgrade, then the DevOps team will help in managing the secret keys in the configuration

Once the builds are upgraded, the designated QA engineer is supposed to do the following-

Load the localization and workflow data as per the release notes, using the standard set of APIs.
Do sanity testing of all the modules. The test cases published as part of the release needs to be used for the qualification. All modules have to be tested using both City A and City B using both English and Hindi. This is to ensure that all the configurations and master data are properly loaded for both tenants.
Before handing over for product sign off, they are expected to test the demo credentials creation utility feature available on the digit website.

Timeline- Upgrade tasks including the QA are supposed to be completed within 5-8 days.

Once the environment is certified by the QA team, it is communicated to the product team for functional sign off.

The product team is expected to do the following-

Check the functional flow of all modules
Check the master data loaded are appropriate
If there are any major bugs or issues uncovered at this stage, this needs to be filed in Jira and the same has to be informed to the responsible parties from the technical team.

Timeline- Product sign off for all modules should not take more than 2-4 days.

Once the product owners sign off, the partnership team is informed about the completion of the task which is then communicated back to the partners. The downtime message from the digit website will be removed by the documentation team. Validate access digit link.

Dos and Don’ts

City A is earmarked to be used by the eGov team, primarily the product team, for giving demos to clients and partners.
City B is to be used by all external stakeholders. When someone registers for user credentials from the website, credentials are created for this user under City B.
No data from City A and B is to be deleted as part of the upgrade.
City A credentials are not to be shared with the external stakeholders
At any point, the demo environment is supposed to be working seamlessly with City A and City B in both English and Hindi.
Any bugs or issues reported by the internal and external team are logged in the engineering Jira account. These bugs will be either fixed in a patch release or the upcoming major release, based on the severity of the bug.
SMS pack to be renewed periodically to ensure the OTP delivery is not impacted at any time.
A retrospective of this entire exercise will be carried out by the responsible folks from the engineering and implementation team who is driving this. This includes the list of slipped defects and the reason for the same. The report of this is shared with all the stakeholders involved in this activity.
No deployment or changes in master data should be done without the knowledge of the product team.
User deletion or deactivation should not happen in CITY A without the knowledge of the product team. Also, any existing user’s details should not be modified without approval.