DEV Community

Discussion on: Ingest NGINX container access logs to ElasticSearch using Fluentd and Docker

Collapse
 
dimastatz profile image
Dima Statz • Edited

Hi Melvin,
thanks a lot for your article.

I am doing pretty similar task - ingesting data to AWS S3 and Apache Kafka by using Nginx and Fluentd. Here you can find more info about it github.com/dimastatz/no-code-data-..., medium.com/swlh/no-code-data-colle....

The questions is - did you tested scenario when Elasticsearch is going down for couple of minutes and then goes up again. Is it possible to prevent data loss in Nginx&Fluentd ?
Thanks,
Dima

Collapse
 
donvitocodes profile image
donvitocodes

Hi Dima, glad you liked the article. The code I did was for a coding test and is not production-ready. So no, I haven't tested it when ElasticSearch is going down.