Impact Emailer

Owned by Vishal Mahuli (Unlicensed)
Feb 05, 2020
4 min read

Objective:

The objective of impact emailer is to showcase the impact of egov’s various products in the state of Punjab. We calculate multiple metrics primarily based on the usage of the application and come up with a impact showcase.

 

Requirements:

  • Prior Knowledge of Java/J2EE

  • Prior Knowledge of SpringBoot

  • Prior Knowledge of PostgresSQL

  • Prior Knowledge of REST APIs and related concepts like path parameters, headers, JSON etc

Prior Knowledge of Kafka and related concepts like Producer, Consumer, Topic etc.

 

Setup:

Step 1: Start the egov-searcher service present in the following repo:

https://github.com/egovernments/core-services/tree/master/egov-searcher

Or port forward the egov-searcher service. 

 

Configurable Properties:

 Following are the properties in application.properties file in impact emailer service which are configurable.

Property

Value

Remarks

egov.impact.emailer.interval.in.secs



604800

It is the time interval, the impact data is subjected to. Currently, we send out weekly impact metrics, that’s why the value is set to ‘1 week in seconds’

egov.impact.emailer.email.to.address



egov-impact-emailer@egovernments.org 

To address of the impact email. It is recommended to create a mailing group including all the recipients and then addressing the impact email to that group. However, the emailer supports comma separated email ids  as well.

egov.resttemplate.timeout.in.ms



300000

Timeout config for resttemplate. The rest calls will only wait for 5 mins for the external service to return. The value should be in ms.

egov.ws.host



https://sunam.lgpunjab.gov.in



Host URL for the W&S application. 

egov.ws.endpoint

/restapi/public/ws/weekly-emailer?ulbCode={ulbCode}&date={date}&interval={interval}

Endpoint to retrieve the W&S data required for the emailer.

egov.ws.ulbcode



2113

ULB Code for retrieving W&S data.

egov.ws.interval.in.ms



604800000

Same as the first property in this list. Difference being the unit, here it is miliseconds.

egov.core.notification.email.topic



egov.core.notification.email



Kafka topic for the egov-notification-email service.

egov.searcher.host

http://localhost:8093



egov-searcher host

egov.searcher.endpoint

/egov-searcher/{moduleName}/{searchName}/_get

Searcher endpoint to fetch data defined under ‘{moduleName}’ and config name ‘{searchName}’

 

Entities:

StateWide: Model to capture the statewide data, currently the following parameters are captured under this metric - ulbCovered, citizenAppDownloads, revenueCollected, servicesApplied, noOfCitizensResgistered.

PT: Model to capture the PT data, currently the following parameters are captured under this metric - ulbCovered, revenueCollected, noOfProperties.

PGR: Model to capture the PGR data, currently the following parameters are captured under this metric - ulbCovered, totalComplaints, redressal, channelBreakup.

TL: Model to capture the TL data, currently the following parameters are captured under this metric - ulbCovered, licenseIssued.

MiscCollections: Model to capture the Miscellaneous Collection data, currently the following parameters are captured under this metric - receiptsGenerated, revenueCollected.

 

How does it work?:

Impact emailer uses egov-searcher to fetch Rainmaker data and an API call to fetch W&S data. The data retrieved through these calls are then processed to calculate different metrics and presented to the end-user in an email, the formatting of which is done using Velocity template engine. 

The impact emailer is a cron job that can be configured to run at a specific time. This run configurations of the emailer lie with Kubernetes. Kube configs decide as to when the emailer job has to be triggered due to which we don’t see timer-tasks or threads in the emailer like we do generally in cron jobs.

According to actual design of the emailer, we had 2 configs subject to time-interval as follows: 

  1. Time interval between the impact data - Right now this interval is set to 1 week, so we fetch data per week. However, with changes in searcher configs, we can set this interval to any range between 1ms to years.

  2. How many such sets to be retrieved - Right now we fetch data for the last 6 weeks. However, this also should be configurable.

For now, we’ve only taken up (1), (2) has been called an over kill for now.

Egov-searcher config for emailer: https://raw.githubusercontent.com/egovernments/configs/master/egov-searcher/weekly-impact-emailer-searcher.yml?token=AE4Z2KD4I72UBPGAPWL44R26IO43K

There are five configs here namely: stateWideData, pgrData, pgrChannelData, ptData, tlData, miscCollections for State Wide data, PGR data, PGR channel breakup data, PT data, TL data and Misc. Collections data respectively. The queries are huge, but if observed carefully you’ll know it is just 6 X one query. The same query is repeated 6 times with different time intervals and later the UNION of all is returned as a data set. This is because the (2) discussed above is fixed to 6 weeks. Observe that, these are JSON queries supported by Postgres. 

The data retrieved, is formatted to the email using Velocity template. This is one of the templating engines actively supported by Java and widely used for sending emails through Java. Velocity template for impact email: https://raw.githubusercontent.com/egovernments/utilities/master/egov-weekly-impact-notifier/src/main/resources/velocity/weeklyimpactflasher.vm?token=AE4Z2KCCGAF7HHUO27L7UDS6IO6K2


Good read about Velocity: https://www.baeldung.com/apache-velocity
Also visit the Apache Velocity project.

An example email:

Interaction Diagram: