This guide aims at enabling the readers to setup a development environment on their system, develop their very own micro-service and integrate/communicate with services running remotely on the sandbox environment.
Prerequisites:
Prior Knowledge of Java/J2EE.
Prior Knowledge of Spring Boot.
Prior Knowledge of REST APIs and related concepts like path parameters, headers, JSON etc.
Prior knowledge of Git
PostgreSQL
Kafka
Following services should be up and running( or else should be pointed to sandbox environment):
User
MDMS
Persister
Location
Localization
Id-Gen
Billing-service
URL-shortener
...
*** NOTE - In case you run into an error stating “error: You must be logged in to the server (Unauthorized)”, try to add sudo before the command. For example, “sudo kubectl get pods”. That should resolve the error.
Swagger Documentation & Generating Project from it:
The first step to develop any micro-service starts at preparing a Swagger documentation which details out all the APIs that the service is going to expose which the clients can consume.
...
| Code Block |
|---|
<dependency> <groupId>org.flywaydb</groupId> <artifactId>flyway-core</artifactId> </dependency> <dependency> <groupId>org.postgresql</groupId> <artifactId>postgresql</artifactId> <version>42.2.2.jre7</version> </dependency> |
Setting up database connection and adding variables in application.properties
All dependent service host urls and API endpoints should be added in application.properties. Along with it whatever properties can be overwritten during deployment should be part of this file(eg: DB url and passwords,Kafka server properties, control knobs for functionalities etc.). To remove boilerplate code for referring variables from application.properties, we create a configuration file and autowire this configuration file wherever we need to refer to these variables. Following properties should be added for configuring database and kafka server( Use the default values, in case you want to tune kafka server that can be overwritten during deployment).
...
To add custom properties in application.properties file and then referencing them in your application -
...
Add SQL scripts to create DB using flyway:
Once the database has been configured, for creating tables in postgres DB we will use flyway. The following properties should be configured in application.properties file to enable flyway migration:
...
| Code Block |
|---|
CREATE TABLE eg_bt_registration(
id character varying(64),
tenantId character varying(64),
applicationNumber character varying(64),
babyFirstName character varying(64),
babyLastName character varying(64),
fatherName character varying(64),
motherName character varying(64),
doctorName character varying(64),
hospitalName character varying(64),
placeOfBirth character varying(64),
dateOfBirth bigint,
createdTime bigint,
lastModifiedTime bigint,
CONSTRAINT uk_eg_bt_registration UNIQUE (id)
);
CREATE TABLE eg_bt_address(
id character varying(64),
tenantId character varying(64),
doorNo character varying(64),
latitude FLOAT,
longitude FLOAT,
buildingName character varying(64),
addressId character varying(64),
addressNumber character varying(64),
type character varying(64),
addressLine1 character varying(256),
addressLine2 character varying(256),
landmark character varying(64),
street character varying(64),
city character varying(64),
locality character varying(64),
pincode character varying(64),
detail character varying(64),
registrationId character varying(64),
createdBy character varying(64),
lastModifiedBy character varying(64),
createdTime bigint,
lastModifiedTime bigint,
CONSTRAINT uk_eg_bt_address PRIMARY KEY (id),
CONSTRAINT fk_eg_bt_address FOREIGN KEY (registrationId) REFERENCES eg_bt_registration (id)
ON UPDATE CASCADE
ON DELETE CASCADE
);
|
Project Structure:
We maintain the following project service for all microservices -
...
Project structure can also be looked at this link - Git Link
Adding MDMS data:
MDMS data is the master data used by the application. This data is stored as JSON on git in the format given below. For any service delivery typical master data will be the allowed values for certain fields and the taxheads (in case if payment is present in the flow).
...
Once data is added to mdms repository, mdms service has to be restarted which will then load the newly added/updated mdms configs. A sample mdms config file can be viewed here - Sample MDMS data file
Adding Workflow configuration:
Workflow configuration should be created based on the business requirements. The configuration can be inserted using the /businessservice/_create API. To create the workflow configuration refer the following documentation: Configuring Workflows For New Product/Entity
...
| Code Block |
|---|
{
"RequestInfo": {
"apiId": "Rainmaker",
"action": "",
"did": 1,
"key": "",
"msgId": "20170310130900|en_IN",
"requesterId": "",
"ts": 1513579888683,
"ver": ".01",
"authToken": "{{devAuth}}"
},
"BusinessServices": [
{
"tenantId": "pb",
"businessService": "BTR",
"business": "birth-services",
"businessServiceSla": 432000000,
"states": [
{
"sla": null,
"state": null,
"applicationStatus": null,
"docUploadRequired": true,
"isStartState": true,
"isTerminateState": false,
"isStateUpdatable": true,
"actions": [
{
"action": "APPLY",
"nextState": "APPLIED",
"roles": [
"CITIZEN",
"EMPLOYEE"
]
}
]
},
{
"sla": null,
"state": "APPLIED",
"applicationStatus": "APPLIED",
"docUploadRequired": false,
"isStartState": false,
"isTerminateState": true,
"isStateUpdatable": false,
"actions": [
{
"action": "APPROVE",
"nextState": "APPROVED",
"roles": [
"EMPLOYEE"
]
},
{
"action": "REJECT",
"nextState": "REJECTED",
"roles": [
"EMPLOYEE"
]
}
]
},
{
"sla": null,
"state": "APPROVED",
"applicationStatus": "APPROVED",
"docUploadRequired": false,
"isStartState": false,
"isTerminateState": false,
"isStateUpdatable": false,
"actions": [
{
"action": "PAY",
"nextState": "REGISTRATIONCOMPLETED",
"roles": [
"SYSTEM_PAYMENT",
"CITIZEN",
"EMPLOYEE"
]
}
]
},
{
"sla": null,
"state": "REJECTED",
"applicationStatus": "REJECTED",
"docUploadRequired": false,
"isStartState": false,
"isTerminateState": true,
"isStateUpdatable": false,
"actions": null
},
{
"sla": null,
"state": "REGISTRATIONCOMPLETED",
"applicationStatus": "REGISTRATIONCOMPLETED",
"docUploadRequired": false,
"isStartState": false,
"isTerminateState": true,
"isStateUpdatable": false,
"actions": null
}
]
}
]
} |
Import core service models:
Models/POJOs of the dependent service can be imported from digit-core-models library (work on creating library is ongoing). These models will be used in integration with the dependent services.
...
| Code Block |
|---|
@Import({TracerConfiguration.class})
public class MainConfiguration {
@Value("${app.timezone}")
private String timeZone;
@PostConstruct
public void initialize() {
TimeZone.setDefault(TimeZone.getTimeZone(timeZone));
}
@Bean
public ObjectMapper objectMapper(){
return new ObjectMapper().disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES).setTimeZone(TimeZone.getTimeZone(timeZone));
}
@Bean
@Autowired
public MappingJackson2HttpMessageConverter jacksonConverter(ObjectMapper objectMapper) {
MappingJackson2HttpMessageConverter converter = new MappingJackson2HttpMessageConverter();
converter.setObjectMapper(objectMapper);
return converter;
}
}
|
Controller Layer:
Controller Layer contains the REST API endpoints which the service wants to expose. The code flow will start from this class. Controller class should be marked with @RestController annotation.
...
| Code Block |
|---|
@Component
public class ResponseInfoFactory {
public ResponseInfo createResponseInfoFromRequestInfo(final RequestInfo requestInfo, final Boolean success) {
final String apiId = requestInfo != null ? requestInfo.getApiId() : "";
final String ver = requestInfo != null ? requestInfo.getVer() : "";
Long ts = null;
if(requestInfo!=null)
ts = requestInfo.getTs();
final String resMsgId = "uief87324";
final String msgId = requestInfo != null ? requestInfo.getMsgId() : "";
final String responseStatus = success ? "successful" : "failed";
return ResponseInfo.builder().apiId(apiId).ver(ver).ts(ts).resMsgId(resMsgId).msgId(msgId).resMsgId(resMsgId)
.status(responseStatus).build();
}
}
|
Service Layer:
Request handlers in the Controller layer call upon the methods defined in the Service layer to perform business logic on the RequestData and prepare the Response to be returned back to the client.
...
ii) For this tutorial, the following Id format has been added as part of this PR - Tutorial Id Format
iii) Now, restart IDGen service and Mdms service and port-forward IDGen service to port 8285 using -
...
| Code Block |
|---|
@Service
public class CalculationService {
public Double calculateLaminationCharges(BirthRegistrationApplication application){
// Add calculation logic according to business requirement
return 10.0;
}
}
|
Repository Layer:
Methods in the service layer, upon performing all the business logic, call methods in the Repository layer to persist or lookup data i.e. it interacts with the configured data store. For executing the queries, JdbcTemplate class is used. JdbcTemplate takes care of creation and release of resources such as creating and closing the connection etc. All database operations namely insert, update, search and delete can be performed on the database using methods of JdbcTemplate class.
...
| Code Block |
|---|
@Slf4j
@Repository
public class BirthRegistrationRepository {
@Autowired
private BirthApplicationQueryBuilder queryBuilder;
@Autowired
private JdbcTemplate jdbcTemplate;
@Autowired
private BirthApplicationRowMapper rowMapper;
public List<BirthRegistrationApplication>getApplications(BirthApplicationSearchCriteria searchCriteria){
List<Object> preparedStmtList = new ArrayList<>();
String query = queryBuilder.getBirthApplicationSearchQuery(searchCriteria, preparedStmtList);
log.info("Final query: " + query);
return jdbcTemplate.query(query, preparedStmtList.toArray(), rowMapper);
}
} |
Producer:
Producer classes help in pushing data from the application to kafka topics. For this, we have a custom implementation of KafkaTemplate class in our tracer library called CustomKafkaTemplate. This implementation of producer class does not change across services of DIGIT. Producer implementation can be viewed here - Producer Implementation
...
| Code Block |
|---|
@Service
@Slf4j
public class Producer {
@Autowired
private CustomKafkaTemplate<String, Object> kafkaTemplate;
public void push(String topic, Object value) {
kafkaTemplate.send(topic, value);
}
} |
Consumers:
Customized SMS creation: Once an application is created/updated the data is pushed on kafka topic. We trigger notification by consuming data from this topic. Whenever any message is consumed the service will call the localisation service to fetch the SMS template. It will then replace the placeholders in the SMS template with the values in the message it consumed(For example: It will replace the {NAME} placeholder with owner name from the data consumed). Once the SMS text is ready, the service will push this data(Create the SMSRequest object which is part of common modules and push the object) on notification topic. (SMS service consumes data from notification topic and triggers SMS).
For our guide, we will be implementing a notification consumer in the following section.
Notification:
Once an application is created/requested or progresses further in the workflow, notifications can be triggered as each of these events are pushed onto kafka topics which can be listened on and a sms/email/in-app notification can be sent to the concerned user(s).
...
| Code Block |
|---|
@Slf4j
@Service
public class NotificationService {
@Autowired
private Producer producer;
@Autowired
private BTRConfiguration config;
@Autowired
private RestTemplate restTemplate;
private static final String smsTemplate = "Dear {NAME}, your birth registration application has been successfully created on the system with application number - {APPNUMBER}.";
public void prepareEventAndSend(BirthRegistrationRequest request){
List<SMSRequest> smsRequestList = new ArrayList<>();
request.getBirthRegistrationApplications().forEach(application -> {
SMSRequest smsRequest = SMSRequest.builder().mobileNumber(application.getApplicantMobileNumber()).message(getCustomMessage(smsTemplate, application)).build();
smsRequestList.add(smsRequest);
});
for (SMSRequest smsRequest : smsRequestList) {
producer.push(config.getSmsNotificationTopic(), smsRequest);
log.info("Messages: " + smsRequest.getMessage());
}
}
private String getCustomMessage(String template, BirthRegistrationApplication application) {
template = template.replace("{APPNUMBER}", application.getApplicationNumber());
template = template.replace("{NAME}", application.getFatherName());
return template;
}
} |
Payment Backupdate:
Once payment is done the application status has to be updated. Since we have a microservice architecture the two services can communicate with each other either through API calls or using message queues(kafka in our case). To avoid any service specific code in collection service we use the second approach to notify the service of payment for its application. Whenever a payment is done the collection service will publish the payment details on a kafka topic. Any microservice which wants to get notified when payments are done can subscribe to this topic. Once the service consumes the payment message it will check if the payment is done for its service by checking the businessService code. If it is done for the given service it will update the application status to PAID or will trigger workflow action PAY depending on the use case.
...
| Code Block |
|---|
@Setter
@Getter
@ToString
@Builder
@NoArgsConstructor
@AllArgsConstructor
@EqualsAndHashCode
public class BillAccountDetail {
@Size(max = 64)
@JsonProperty("id")
private String id;
@Size(max = 64)
@JsonProperty("tenantId")
private String tenantId;
@Size(max = 64)
@JsonProperty("billDetailId")
private String billDetailId;
@Size(max = 64)
@JsonProperty("demandDetailId")
private String demandDetailId;
@JsonProperty("order")
private Integer order;
@JsonProperty("amount")
private BigDecimal amount;
@JsonProperty("adjustedAmount")
private BigDecimal adjustedAmount;
@JsonProperty("isActualDemand")
private Boolean isActualDemand;
@Size(max = 64)
@JsonProperty("taxHeadCode")
private String taxHeadCode;
@JsonProperty("additionalDetails")
private JsonNode additionalDetails;
@JsonProperty("auditDetails")
private AuditDetails auditDetails;
} |
Persister configurations:
The persister configuration is written in a YAML format. The INSERT and UPDATE queries for each table is added in prepared Statement format, followed by the jsonPaths of values which has to be inserted/updated.
...
| Code Block |
|---|
serviceMaps:
serviceName: btr-services
mappings:
- version: 1.0
description: Persists birth details in tables
fromTopic: save-bt-application
isTransaction: true
queryMaps:
- query: INSERT INTO eg_bt_registration(id,tenantid,applicationNumber,babyFirstName,babyLastName,fatherName,motherName,doctorName,hospitalName,placeOfBirth,dateOfBirth,createdtime, lastmodifiedtime) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?);
basePath: BirthRegistrationApplications.*
jsonMaps:
- jsonPath: $.BirthRegistrationApplications.*.id
- jsonPath: $.BirthRegistrationApplications.*.tenantId
- jsonPath: $.BirthRegistrationApplications.*.applicationNumber
- jsonPath: $.BirthRegistrationApplications.*.babyFirstName
- jsonPath: $.BirthRegistrationApplications.*.babyLastName
- jsonPath: $.BirthRegistrationApplications.*.fatherName
- jsonPath: $.BirthRegistrationApplications.*.motherName
- jsonPath: $.BirthRegistrationApplications.*.doctorName
- jsonPath: $.BirthRegistrationApplications.*.hospitalName
- jsonPath: $.BirthRegistrationApplications.*.placeOfBirth
- jsonPath: $.BirthRegistrationApplications.*.dateOfBirth
- jsonPath: $.BirthRegistrationApplications.*.auditDetails.createdTime
- jsonPath: $.BirthRegistrationApplications.*.auditDetails.lastModifiedTime
- query: INSERT INTO eg_bt_address(id, tenantid, doorno, latitude, longitude, buildingname, addressid, addressnumber, type, addressline1, addressline2, landmark, street, city, locality, pincode, detail, registrationid, createdby, lastmodifiedby, createdtime, lastmodifiedtime) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?);
basePath: BirthRegistrationApplications.*
jsonMaps:
- jsonPath: $.BirthRegistrationApplications.*.address.id
- jsonPath: $.BirthRegistrationApplications.*.address.tenantId
- jsonPath: $.BirthRegistrationApplications.*.address.doorNo
- jsonPath: $.BirthRegistrationApplications.*.address.latitude
- jsonPath: $.BirthRegistrationApplications.*.address.longitude
- jsonPath: $.BirthRegistrationApplications.*.address.buildingName
- jsonPath: $.BirthRegistrationApplications.*.address.addressId
- jsonPath: $.BirthRegistrationApplications.*.address.addressNumber
- jsonPath: $.BirthRegistrationApplications.*.address.type
- jsonPath: $.BirthRegistrationApplications.*.address.addressLine1
- jsonPath: $.BirthRegistrationApplications.*.address.addressLine2
- jsonPath: $.BirthRegistrationApplications.*.address.landmark
- jsonPath: $.BirthRegistrationApplications.*.address.street
- jsonPath: $.BirthRegistrationApplications.*.address.city
- jsonPath: $.BirthRegistrationApplications.*.address.locality.name
- jsonPath: $.BirthRegistrationApplications.*.address.pincode
- jsonPath: $.BirthRegistrationApplications.*.address.detail
- jsonPath: $.BirthRegistrationApplications.*.address.registrationId
- jsonPath: $.BirthRegistrationApplications.*.address.createdBy
- jsonPath: $.BirthRegistrationApplications.*.address.lastModifiedBy
- jsonPath: $.BirthRegistrationApplications.*.address.createdTime
- jsonPath: $.BirthRegistrationApplications.*.address.lastModifiedTime
- version: 1.0
description: Update birth registration applications in table
fromTopic: update-bt-application
isTransaction: true
queryMaps:
- query: UPDATE eg_bt_registration SET tenantid = ?,babyFirstName = ?, dateOfBirth = ? WHERE id=?;
basePath: BirthRegistrationApplications.*
jsonMaps:
- jsonPath: $.BirthRegistrationApplications.*.tenantId
- jsonPath: $.BirthRegistrationApplications.*.babyFirstName
- jsonPath: $.BirthRegistrationApplications.*.dateOfBirth
- jsonPath: $.BirthRegistrationApplications.*.id |
Indexer Configuration:
Indexer is designed to perform all the indexing tasks of the digit platform. The service reads records posted on specific kafka topics and picks the corresponding index configuration from the yaml file provided by the respective module configuration. Configurations are yaml based. Detailed guide to create indexer configs are mentioned in the following document - Indexer Configuration Guide .
...
| Code Block |
|---|
ServiceMaps:
serviceName: Birth Registration Service
version: 1.0.0
mappings:
- topic: save-bt-application
configKey: INDEX
indexes:
- name: btindex-v1
type: general
id: $.id
isBulk: true
timeStampField: $.auditDetails.createdTime
jsonPath: $.BirthRegistrationApplications
customJsonMapping:
indexMapping: {"Data":{"birthapplication":{},"history":{}}}
fieldMapping:
- inJsonPath: $
outJsonPath: $.Data.birthapplication
externalUriMapping:
- path: http://localhost:8282/egov-workflow-v2/egov-wf/process/_search
queryParam: businessIds=$.applicationNumber,history=true,tenantId=$.tenantId
apiRequest: {"RequestInfo":{"apiId":"org.egov.pt","ver":"1.0","ts":1502890899493,"action":"asd","did":"4354648646","key":"xyz","msgId":"654654","requesterId":"61","authToken":"d9994555-7656-4a67-ab3a-a952a0d4dfc8","userInfo":{"id":1,"uuid":"1fec8102-0e02-4d0a-b283-cd80d5dab067","type":"EMPLOYEE","tenantId":"pb.amritsar","roles":[{"name":"Employee","code":"EMPLOYEE","tenantId":"pb.amritsar"}]}}}
uriResponseMapping:
- inJsonPath: $.ProcessInstances
outJsonPath: $.Data.history
- topic: update-bt-application
configKey: INDEX
indexes:
- name: btindex-v1
type: general
id: $.id
isBulk: true
timeStampField: $.auditDetails.createdTime
jsonPath: $.BirthRegistrationApplications
customJsonMapping:
indexMapping: {"Data":{"tradelicense":{},"history":{}}}
fieldMapping:
- inJsonPath: $
outJsonPath: $.Data.tradelicense
externalUriMapping:
- path: http://localhost:8282/egov-workflow-v2/egov-wf/process/_search
queryParam: businessIds=$.applicationNumber,history=true,tenantId=$.tenantId
apiRequest: {"RequestInfo":{"apiId":"org.egov.pt","ver":"1.0","ts":1502890899493,"action":"asd","did":"4354648646","key":"xyz","msgId":"654654","requesterId":"61","authToken":"d9994555-7656-4a67-ab3a-a952a0d4dfc8","userInfo":{"id":1,"uuid":"1fec8102-0e02-4d0a-b283-cd80d5dab067","type":"EMPLOYEE","tenantId":"pb.amritsar","roles":[{"name":"Employee","code":"EMPLOYEE","tenantId":"pb.amritsar"}]}}}
uriResponseMapping:
- inJsonPath: $.ProcessInstances
outJsonPath: $.Data.history
|
Certificate Generation:
The final step in this process is the creation of configs to create a birth registration PDF for the citizens to download. For this, we will make use of DIGIT’s PDF service which uses PDFMake and Mustache libraries to generate PDF. A detailed documentation on PDF service and generating PDFs using PDF service can be found here - PDF Generation Service.
...