How to set up an index alias that points to multiple indices in Elasticsearch

Index aliasing is a powerful feature in Elasticsearch that gives us the ability to control the indices. One could create as many indices as possible and use an alias to point to the right index or one can point the index alias to all indices, depending on your application needs.

Scenarios

Imagine if you had the following scenarios when working with Elasticsearch for your application needs:

• Scenario 1: A snapshot of your index that needs to be restored to a new cluster, however, you forgot to restore it, and instead, you created a new index. As time passed, you realised that new data had amassed in the newly created index and you needed to combine the data from the snapshot with the new index.
• Scenario 2: You have an index that requires certain schema changes or re-indexing. To be able to ensure your users are able to search from your instance with almost no downtime, set up a new index with new schema changes and re-index the data.

Introducing multi-index aliasing. You can create a single index alias that points to multiple indices. There's an endpoint from Elasticsearch that allows you to perform one or more alias actions in a single atomic operation.

Prerequisites

• Verify if the alias you'd like to use has not been used yet by using the _alias/alias-name endpoint - HEAD _alias/my-alias
  • If it returns a 404, it means the alias is available and can be used in setting up the multi-index aliasing
  • If it returns a 200, it means it's being used and you need to choose another alias
• Verify your indices are readily available within your cluster/instance.

Steps

1. Ensure that your old snapshot is ready (if you don't have your secondary index setup).
2. When you do restore the snapshot index, ensure that the index name is different than the newly created to avoid conflicts.
   • Clone the index from the old snapshot/cluster if you have to
3. Run the following POST request to your designated cluster:

POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "new_index",
        "alias": "new_index_alias"
      }
    },
    {
      "add": {
        "index": "old_index",
        "alias": "new_index_alias"
      }
    }
  ]
}

But wait, what if I want to specify the new data is written to a specific index?

You can pass in the **is_write_index** parameter in the payload as follows:

POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "new_index",
        "alias": "new_index_alias",
        "is_write_index": true
      }
    },
    {
      "add": {
        "index": "old_index",
        "alias": "new_index_alias"
      }
    }
  ]
}

As you can see, it's fairly straightforward to set up an index alias that points to multiple indices within your Elasticsearch instance.

If you want to query data from the alias, simply perform the following:

GET /new_index_alias/_search

This would help you query across two indices new_index and old_index.

Benefits

• Zero downtime: Bring up a new index with new changes, point alias to new index when it's ready. Keep the old index stale, in case it's needed for future reference or just as a backup.
• Multiple Indices: We want to be able to query multiple indices? Then create an alias that points to multiple indices.

Resources

• https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-aliases.html
• https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-alias-exists.html

Comments

[Ashish Mishra]

Is it possible to write to alias index, which inturn write to both the indexes ?