Promotion Steps:
Promote egovio/property-services:141-PT_SERVICES_FEATURE-2bb36a8a-db:v1.1.2-ba1d2417-170
Add https://github.com/egovernments/configs/blob/master/egov-indexer/property-services.yml to indexer
...
Index for properties (“property-services”)
It will Index property records
The records will be picked from following kafka topics
save-property-registry :- The records on this topic are for newly created properties
update-property-registry :- The records on this topic are for any update on existing property
cancel-property-registry :- Records for cancelled properties
property-registry-legacyIndex :- This topic is used for reindexing of old records
Every time any new property is created or existing property is updated new record is pushed on elastic search index. Please note we are never updating any record on index, each request for indexing will be pushed as new record, if existing property is updated then the existing record on index is updated. So this index stores latest information about any propertyId
As key for indexed records we are using combination of propertyId,tenantId and last modified time fields of the property
Property last modified time is used as timestamp
...
Index for properties assessments (“property-assessments”)
It will Index property assessment records
The records will be picked from following kafka topics
save-pt-assessment :- The records on this topic are for newly created property assessments
update-pt-assessment :- The records on this topic are for any update on existing assessments
cancel-property-assessment :- Records for cancelled assessments
pt-assessment-legacyIndex :- This topic used for reindexing of old records
As key for indexed records we are using combination of assessment number and tenantId
On update or cancel assessment existing record on index is modified, so we are keeping only latest information about any assessment.
Assessment created time is used as timestamp
...
All the PIIs are removed from Index
On creation of new property, new record would be pushed on property index on each property updateand on creation of new assessment, new record would be pushed on assessment index will be created. For assessment index on update existing record on index will be updated. And on cancellation, updates etc existing record would be updated on both indexes.
In earlier case property details were kept inside “propertydetail” in property object, but now this field is removed and information kept inside it is part of property object itself
Earlier if the user updates property information the same new information was saved in “propertydetail” array along with old property information entries, but now there will be two separate records on index one for old property and other for updated propertyEarlier same property object used to contain property history now for each property update new record is createdwe are not storing any property history and both records on property and assessment index will keep latest information about property and assessment respectively.
Earlier property object itself used to contain assessment information, but now assessment information is separated and kept on separate assessment index
Records on both property and assessment index contains workflow information
Few fields are modified and added in the property object, for more information please refer “Property” object in property service API contract https://github.com/egovernments/municipal-services/blob/master/docs/property-services/property-services.yml
...
The structure of ward and tenantData is as before
Impact on Dashboard:
For taking any count on property index duplicate properties should be removed because if the property was updated same property can have have more than one recordTotal count of properties in the system would be total number of records on property index
For counting assessments please use assessment index, the records includes assessment number and financial year fields which are being used in current Dashboard
Get “usageCategoryMajor” from property object instead of propertydetail, when searching only latest value of this field would be available for a property, search all records with given propertyidany property
For all fields which were present in “propertydetail” except assessment information get them from property object
Assessment index records contain propertyId field which can be used to get count of total assessment for a property from assessment index
Property Services: Tenant and date range wise re-indexing Utility:
Need to have criteria based re-indexing because in any production environments we would not be doing or needing entire data reindexing
It would be for either for one tenant or for certain criteria like all residential properties
Added a plainsearch API endpoints for multiple tenantIds and date range wise reindexing
fromDate() and toDate() are to be converted to respective unix timestamps.
Query parameters are tenantId(can be single or multiple), fromDate and toDate which are to be converted to respective unix timestamps.
Basically Unix timestamp is a method to track time as running total of seconds. The count starts from Jan 1st 1970, which is considered as Unix Epoch. Unix timestamp is the number of seconds elapsed between a given date and unix epoch.
Reindexing commands:
Property Index
...
"uri":"http://property-services:8080/property-services/property/_plainauditsearchplainsearch",
"tenantIdForOpenSearch":"pb", "paginationDetails":{
"offsetKey":"offset",
...
"uri":"http://property-services:8080/property-services/assessment/_plainsearch",
"tenantIdForOpenSearch":"pb ", "paginationDetails":{
"offsetKey":"offset",
...
"legacyIndexTopic":"pt-assessment-legacyIndex",
"tenantId":"pb.amritsar"
}'
Tenant and Date wise Indexing
curl -X POST \
http://localhost:8280/property-services/property/_plainsearch?fromDate=25112020&toDate=26112020&tenantId=pb.amritsar \
H 'Cache-Control: no-cache' \
H 'Content-Type: application/json' \
d '{
"RequestInfo": {
"apiId": "Rainmaker",
"ver": ".01",
"ts": null,
"action": "search",
"did": "1",
"key": "",
"msgId": "20170310130900|en_IN",
"authToken": null,
"correlationId": "654938aa-ed8a-4d8f-8bd9-a2b73db9cbfe",
"userInfo": {
"id": 24710,
"userName": "EMP1",
"name": "EMP ONE",
"type": "EMPLOYEE",
"mobileNumber": "8888888888",
"emailId": "emp1@egovernments.org",
"roles": [
{
"id": null,
"name": "WS Counter Employee",
"code": "WS_CEMP",
"tenantId": "pb.amritsar"
},
{
"id": null,
"name": "WS Document Verifier",
"code": "WS_DOC_VERIFIER",
"tenantId": "pb.amritsar"
},
{
"id": null,
"name": "Employee",
"code": "EMPLOYEE",
"tenantId": "pb.amritsar"
}
],
"tenantId": "pb.amritsar",
"uuid": "b81e378d-0bcf-4ea9-a7eb-d24163c74b8e"
}
}
}
}'