Overview
Due to certain limitations of Druid and Metabase, we decided to move the IFIX iFIX DWSS dashboard to ElasticSearch/DSS. As a consequence of this, we had to add a couple of new services like -
IFIX iFIX Migration Toolkit - This utility service helps in moving fiscal data from MongoDB to ElasticSearch.
IFIX iFIX ES Pipeline - This pipeline service listens onto fiscal data(both existing and new ones), enriches them, performs certain pre-computations so as to make writing dashboard queries efficient and simple to write.
Indexer Service - Indexer service was deployed to index fiscal records onto ElasticSearch.
Persister Service - Persister service was deployed to persist migration progress entries which we are maintaining so as to avoid duplication of records. Persister in turn stores records in PostgreSQL database.
Dashboard Analytics Service - Dashboard analytics service was deployed to create charts out of fiscal data which was moved to ElasticSearch.
DIGIT-UI - A customized version of DIGIT-UI was deployed in order to look at the new dashboard.
...