DEV Community

Sergejs L
Sergejs L

Posted on

Note-to-future-self: k8s /metrics hanging

Context

Clean k8s cluster, just after init was made.
All auth disabled in config.yml:

authentication:
  anonymous:
    enabled: true
  webhook:
    enabled: false
authorization:
  mode: AlwaysAllow
Enter fullscreen mode Exit fullscreen mode
curl -k https://127.0.0.1:10250/healthz responds with ok 
Enter fullscreen mode Exit fullscreen mode

Problem

These curls hanging:

curl -k https://127.0.0.1:10250/metrics 
curl -k https://127.0.0.1:10250/stats/summary 
Enter fullscreen mode Exit fullscreen mode

Solution

sudo tee /etc/systemd/system/kubelet.service.d/99-metricsfix.conf <<'EOF'
[Service]
Environment="KUBELET_EXTRA_ARGS=--enable-cadvisor-endpoints=false --feature-gates=PodAndContainerStatsFromCRI=true --volume-stats-agg-interval=0 --housekeeping-interval=10m --kubelet-cgroups=/system.slice/kubelet.service"
EOF
sudo systemctl daemon-reload
sudo systemctl restart kubelet
Enter fullscreen mode Exit fullscreen mode

Top comments (0)