ES-Curator

Objective

Installing Curator

Simple use pip install elasticsearch-curator to install Curator on your machine. I prefer to install it on the Elasticsearch machine itself.

Configuring Curator

Create a file curator.yml with following contents.

---client:hosts:-127.0.0.1port:9200url_prefix:use_ssl:Falsecertificate:client_cert:client_key:ssl_no_validate:Falsehttp_auth:timeout:30master_only:Falselogging:loglevel:INFOlogfile:logformat:defaultblacklist:['elasticsearch','urllib3']

Now, we need to define an action. i.e. What will curator do. There are many actions to choose from. Check the documentation for more information

For this dicussion, we will use Delete Indices as the action, since this is what we want to do.

Below is the sample action file delete_indices.yml , which will delete the logstash indices which are older than 10 days.

---actions:1:action:delete_indicesdescription:>-Delete indices older than 45 days (based on index name), for logstash-prefixed indices. Ignore the error if the filter does not result in anactionable list of indices (ignore_empty_list) and exit cleanly.options:ignore_empty_list: Truetimeout_override:continue_if_exception: Falsedisable_action: Falsefilters:- filtertype: patternkind: prefixvalue: logstash-exclude:- filtertype: agesource: namedirection: oldertimestring: '%Y.%m.%d'unit: daysunit_count: 10exclude:

To run this action, simple use the command

curator ./delete_index.yml --config ./curator.yml --dry-run 2017-04-09 17:27:46,075 INFO Preparing Action ID: 1, "delete_indices" 2017-04-09 17:27:46,080 INFO Trying Action ID: 1, "delete_indices": Delete indices older than 45 days (based on index name), for logstash- prefixed indices. Ignore the error if the filter does not result in an actionable list of indices (ignore_empty_list) and exit cleanly. 2017-04-09 17:27:46,538 INFO DRY-RUN MODE. No changes will be made. 2017-04-09 17:27:46,538 INFO (CLOSED) indices may be shown that may not be acted on by action "delete_indices". 2017-04-09 17:27:46,538 INFO Action ID: 1, "delete_indices" completed. 2017-04-09 17:27:46,538 INFO Job completed.

The --dry-run mode will not actually delete the index. It can be used to test the output of the action.

If you want to schedule it in a cron, you can do so using crontab -e

00 8 * * * root curator /path/delete_index.yml --config /path/curator.yml

The above configuration will cleanup the indices older than 10 days everyday at 8 AM.

Success metrics

Requirements

User interaction and design

Open Questions

Out of Scope