DEV Community

Sandeep Kanabar
Sandeep Kanabar

Posted on

Benefits of time-based indices on reindex, alias, ilm

In addition to the benefits listed here, using time-based indices helps in the following:

  1. Avoids having to reindex entire data
  2. Efficient Deletion and application of ILM
  3. Easy to include / exclude indices based on alias
1. Avoids having to reindex entire data

If the data influx increases, we could easily set "number_of_shards": 3 in the index template and this would get effected for tomorrow's day-wise index. Without the need to reindex any data, the number of shards could be easily changed.

2. Efficient Deletion and application of ILM

Let's say we need to retain data upto 90 days. Thus, for a day-wise index which is older than 90 days, that entire index can be purged / deleted. This is far more efficient than purging older records from indices which makes them pretty un-optimised from search perspective.
Also, index lifecycle management becomes simplified with time-based indices.

3. Easy to include / exclude indices based on alias

Let's assume the cluster needs to retain 90 days data but needs to search only on the last 60 days data. Alias to the rescue. In this case, define an alias in index template that gets mapped to newly created day-wise indices. As soon as a past index becomes older than 60 days, the alias is removed from that index. This ensures that at any given point of time, the alias will point to a maximum of 60 day-wise indices.

Top comments (1)

Collapse
 
Sloan, the sloth mascot
Comment deleted