...
Will increase the development time and efforts
Refreshing the configuration:
Once the configuration is pulled by git-sync the services should be notified to reload the configuration. Git-sync provides webhooks which are called once the new hash is synced. We will add one /config/_refresh API in each of these services and configure them in git-sync. Whenever git-sync pulls data successfully it will trigger _refresh API which will then refresh the configuration.
...
Since the _refresh API call goes to only one pod, only that pod will refresh the data. Because of this we cannot use local cache. We will need a cluster-aware caching implementation, which we can do using the Redis instance that is already deployed. By moving the configuration to redis, even if one pod refreshes the cache it will reflect in all the pods for that service.
For services like indexer and persister which load consumers from configuration, cache burst won’t work. We will have to use configuration versioning. In cache we will store configuration along with the hash associated with that version. Services like indexer and persister will maintain a variable called localOperatingConfigHash in its heap memory. Whenever it fetches data from redis cache it will compare the localOperatingConfigHash with the current hash(version) of the configuration, if it matches it will go ahead and perform the process, but if it is not same it will reload the consumers from the new configuration and update the localOperatingConfigHash to the value it got from redis cache
...
Resource Optimisation:
Currently services like persister, indexer, etc. use a git-sync container to load configuration files.If we run them as sidecar containers, each service will require one. We can remove this container from all these services by enhancing the MDMS service. This service(s) will now load configuration files by calling MDMS service. MDMS service will return the requested configuration files. Using this approach only MDMS service will be required to run the git-sync pod.
...