Sometimes you might get
Error: disk usage exceeded flood-stage watermark, index has read-only-allow-delete block
or
ClusterBlockException[blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]
This error indicates a data node is critically low on disk space and has reached the flood-stage disk usage watermark. To prevent a full disk, when a node reaches this watermark, Elasticsearch blocks writes to any index with a shard on the node. If the block affects related system indices, Kibana and other Elastic Stack features may become unavailable.
This means you'll only be able to read the Indices, Delete the Indices but cannot write new Indices.
Get the Cluster Settings
curl elasticsearch-sc:9200/_settings?pretty
You'll see the read_only_allow_delete following as true for the Indices that means the cluster is in read-only mode
"blocks" : {
"read_only_allow_delete" : "true"
},
After the Disk Cleanup, When you have deleted the non-required Indices or after adding more space to the cluster, The Cluster needs to be put back to the writable mode
Putting Cluster back to Writable Mode
curl -XPUT -H "Content-Type: application/json" \
http://elasticsearch-sc:9200/_all/_settings \
-d '{"index.blocks.read_only_allow_delete": null}'
You can check the cluster settings again to confirm the change
Oldest comments (0)