...
The handler will contain two APIs, _create and _update. The _create API will be used to add configuration for the first time. It will work in idempotent manner i.e if any configuration is already present it will ignore it (won’t add that particular config). Modules can integrate with this _create API. During application initialisation, the service will call this API to add the default configurations(if required). For any further changes to configuration the developer can call the _update API.
Design:
...
Moving Config to MDMS:
Currently services like persister, indexer, etc. use a git-sync container to load configuration files.If we run them as sidecar containers, each service will require one. We can remove this container from all these services by enhancing the MDMS service. This service(s) will now load configuration files by calling MDMS service. MDMS service will return the requested configuration files. Using this approach only MDMS service will be required to run the git-sync pod.
Note:
We will keep the existing resource loader logic to load from the resource folder. In addition to that we will provide another type of resource loader which will fetch from MDMS. Based on the environment variable it will select the loader.
Currently MDMS data is of JSON format. We need to add support(if not present) for YAML files to serve the configuration from MDMS.(Another way is to change the configuration format from YAML to JSON)
Design:
...
The configuration handler will integrate with Git and the required DIGIT services (workflow, localisation etc.) to add/update configuration. Once the configuration is added in Git, the pods using this configuration should pull the latest changes. There are multiple approaches for achieving this. Below are the list of approaches with their pros and cons.
...
Will increase the development time and efforts
Refreshing the configuration:
Once the configuration is pulled by git-sync the services should be notified to reload the configuration. Git-sync provides webhooks which are called once the new hash is synced. We will add one /config/_refresh API in each of these services and configure them in git-sync. Whenever git-sync pulls data successfully it will trigger _refresh API which will then refresh the configuration.
...
Since the _refresh API call goes to only one pod, only that pod will refresh the data. Because of this we cannot use local cache. We will need a cluster-aware caching implementation, which we can do using the Redis instance that is already deployed. By moving the configuration to redis, even if one pod refreshes the cache it will reflect in all the pods for that service.
Resource Optimisation:
Currently For services like indexer and persister , indexer, etc. use a git-sync container to load configuration files.If we run them as sidecar containers, each service will require one. We can remove this container from all these services by enhancing the MDMS service. This service(s) will now load configuration files by calling MDMS service. MDMS service will return the requested configuration files. Using this approach only MDMS service will be required to run the git-sync pod.
Note:
...
We will keep the existing resource loader logic to load from the resource folder. In addition to that we will provide another type of resource loader which will fetch from MDMS. Based on the environment variable it will select the loader.
...
which load consumers from configuration, cache burst won’t work. We will have to use configuration versioning. In cache we will store configuration along with the hash associated with that version. Services like indexer and persister will maintain a variable called localOperatingConfigHash in its heap memory. Whenever it fetches data from redis cache it will compare the localOperatingConfigHash with the current hash(version) of the configuration, if it matches it will go ahead and perform the process, but if it is not same it will reload the consumers from the new configuration and update the localOperatingConfigHash to the value it got from redis cache
...
Another optimised approach will be that persiter service fetches the configs when _refresh API is called. After fetching the config the service will get all the topics present in the configs and get a hash of that list of topics. This hash service will set in its local variable and add it in redis along with the configs. When another pod fetches the config and hash from redis it will compare the hash with its local hash and based on that it will restart consumers. In this way consumers will be only restarted when topics are added or removed.
Tracer Enhancements:
We will update the tracer library to automatically connect with the config handler module during initialisation. The tracer module will work by using a variable called tracer.config.path. This variable will contain the path where the configuration json is present. By default it will point to the resource folder inside the code base, but for production use cases it can be overwritten with Git url or any other valid file path as per requirement.
...