Impact Emailer
- Vishal Mahuli (Unlicensed)
Objective:
The objective of impact emailer is to showcase the impact of egov’s various products in the state of Punjab. We calculate multiple metrics primarily based on the usage of the application and come up with a impact showcase.
Requirements:
Prior Knowledge of Java/J2EE
Prior Knowledge of SpringBoot
Prior Knowledge of PostgresSQL
Prior Knowledge of REST APIs and related concepts like path parameters, headers, JSON etc
Prior Knowledge of Kafka and related concepts like Producer, Consumer, Topic etc.
Setup:
Step 1: Start the egov-searcher service present in the following repo:
https://github.com/egovernments/core-services/tree/master/egov-searcher
Or port forward the egov-searcher service.
Configurable Properties:
Following are the properties in application.properties file in impact emailer service which are configurable.
Property | Value | Remarks |
egov.impact.emailer.interval.in.secs | 604800 | It is the time interval, the impact data is subjected to. Currently, we send out weekly impact metrics, that’s why the value is set to ‘1 week in seconds’ |
egov.impact.emailer.email.to.address | To address of the impact email. It is recommended to create a mailing group including all the recipients and then addressing the impact email to that group. However, the emailer supports comma separated email ids as well. | |
egov.resttemplate.timeout.in.ms | 300000 | Timeout config for resttemplate. The rest calls will only wait for 5 mins for the external service to return. The value should be in ms. |
egov.ws.host | Host URL for the W&S application. | |
egov.ws.endpoint | /restapi/public/ws/weekly-emailer?ulbCode={ulbCode}&date={date}&interval={interval} | Endpoint to retrieve the W&S data required for the emailer. |
egov.ws.ulbcode | 2113 | ULB Code for retrieving W&S data. |
egov.ws.interval.in.ms | 604800000 | Same as the first property in this list. Difference being the unit, here it is miliseconds. |
egov.core.notification.email.topic | egov.core.notification.email | Kafka topic for the egov-notification-email service. |
egov.searcher.host | egov-searcher host | |
egov.searcher.endpoint | /egov-searcher/{moduleName}/{searchName}/_get | Searcher endpoint to fetch data defined under ‘{moduleName}’ and config name ‘{searchName}’ |
Entities:
StateWide: Model to capture the statewide data, currently the following parameters are captured under this metric - ulbCovered, citizenAppDownloads, revenueCollected, servicesApplied, noOfCitizensResgistered.
PT: Model to capture the PT data, currently the following parameters are captured under this metric - ulbCovered, revenueCollected, noOfProperties.
PGR: Model to capture the PGR data, currently the following parameters are captured under this metric - ulbCovered, totalComplaints, redressal, channelBreakup.
TL: Model to capture the TL data, currently the following parameters are captured under this metric - ulbCovered, licenseIssued.
MiscCollections: Model to capture the Miscellaneous Collection data, currently the following parameters are captured under this metric - receiptsGenerated, revenueCollected.
How does it work?:
Impact emailer uses egov-searcher to fetch Rainmaker data and an API call to fetch W&S data. The data retrieved through these calls are then processed to calculate different metrics and presented to the end-user in an email, the formatting of which is done using Velocity template engine.
The impact emailer is a cron job that can be configured to run at a specific time. This run configurations of the emailer lie with Kubernetes. Kube configs decide as to when the emailer job has to be triggered due to which we don’t see timer-tasks or threads in the emailer like we do generally in cron jobs.
According to actual design of the emailer, we had 2 configs subject to time-interval as follows:
Time interval between the impact data - Right now this interval is set to 1 week, so we fetch data per week. However, with changes in searcher configs, we can set this interval to any range between 1ms to years.
How many such sets to be retrieved - Right now we fetch data for the last 6 weeks. However, this also should be configurable.
For now, we’ve only taken up (1), (2) has been called an over kill for now.
Egov-searcher config for emailer: https://raw.githubusercontent.com/egovernments/configs/master/egov-searcher/weekly-impact-emailer-searcher.yml?token=AE4Z2KD4I72UBPGAPWL44R26IO43K
There are five configs here namely: stateWideData, pgrData, pgrChannelData, ptData, tlData, miscCollections for State Wide data, PGR data, PGR channel breakup data, PT data, TL data and Misc. Collections data respectively. The queries are huge, but if observed carefully you’ll know it is just 6 X one query. The same query is repeated 6 times with different time intervals and later the UNION of all is returned as a data set. This is because the (2) discussed above is fixed to 6 weeks. Observe that, these are JSON queries supported by Postgres.
The data retrieved, is formatted to the email using Velocity template. This is one of the templating engines actively supported by Java and widely used for sending emails through Java. Velocity template for impact email: https://raw.githubusercontent.com/egovernments/utilities/master/egov-weekly-impact-notifier/src/main/resources/velocity/weeklyimpactflasher.vm?token=AE4Z2KCCGAF7HHUO27L7UDS6IO6K2
Good read about Velocity: https://www.baeldung.com/apache-velocity
Also visit the Apache Velocity project.
An example email:
Interaction Diagram:
