The latest articles on DEV Community by Sumit Vohra (@sumit_vohra_09e9b7e4fac4b). https://dev.to/sumit_vohra_09e9b7e4fac4b https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F2455408%2F2dfb8541-6710-41d1-b078-a4ed0bb55291.jpg https://dev.to/sumit_vohra_09e9b7e4fac4b en Sumit Vohra Tue, 19 Nov 2024 15:46:35 +0000 https://dev.to/sumit_vohra_09e9b7e4fac4b/otel-collector-issues-over-short-and-long-term--4b05 https://dev.to/sumit_vohra_09e9b7e4fac4b/otel-collector-issues-over-short-and-long-term--4b05 <p>Hey community,<br> I have been using otel-collector for my org ( x Tbs/day ) observability in k8s setup for sometime. Following is my experience. Do you had the same or was it different and how did you overcome it?</p> <p><strong>Long Term ( 6 months + of using ) :</strong></p> <ol> <li><p>Poor data-loss detecting capabilities. I have been loosing data but no good way to see that. Agent/collector pods prints error logs but since pipeline doesn't work so it doesn't reach the log-system</p></li> <li><p>No UI to view/monitor my existing connections and pick and drop functionalities</p></li> <li><p>No easy way to inject transformers, for example i need to change format of some data for SIEM/snowflake, drop some log data like cribl, i should be able to do it within otel itself.</p></li> </ol> <p><strong>Short term ( while setup ) :</strong></p> <ol> <li><p>No grpc-native load balancer in otel. Horizontal scaling became an issue, as the agent runs on grpc and owing to no native grpc-load balancer directly operating over otel, resulted in oversizing my clusters unnecessarily.</p></li> <li><p>Distributed tracing needs more automation, i had to manually stitch at various places.</p></li> </ol> <p>Anyone else faced similar issues or others???</p> devops observability