Hi Tim, We have a Flafka cluster with Kafka-Ambari-Zookeeper-Flume-RabbitMQ-HDFS Clouderra stack, we had a electric surge a month ago, after that incident, we lost some blocks in HDF with bad, but now Flume fails to deliver the same count of topic partition messages to HDFS, there is no lag or offset difference, but balance of messages flown from Kafka is dissipated in Flume sinks or lost without reaching HDFS destination successfully, flume dissolving messages in the headband. We are having problems sinking hdfs. When it restarts, it corrects, but after a certain time flume cannot write.
Hi Tim, We have a Flafka cluster with Kafka-Ambari-Zookeeper-Flume-RabbitMQ-HDFS Clouderra stack, we had a electric surge a month ago, after that incident, we lost some blocks in HDF with bad, but now Flume fails to deliver the same count of topic partition messages to HDFS, there is no lag or offset difference, but balance of messages flown from Kafka is dissipated in Flume sinks or lost without reaching HDFS destination successfully, flume dissolving messages in the headband. We are having problems sinking hdfs. When it restarts, it corrects, but after a certain time flume cannot write.
dev-to-uploads.s3.amazonaws.com/i/...