Requirements for this POC
This POC will use Multipass to create instances to install Grafana, Prometheus and Microk8s, we will install Metrics Server, Kube-State-Metrics and Istio + Basic Prometheus and then we will make metrics used by our main instance with Prometheus.
Install Multipass
sudo snap install multipass --classic
MicroK8s
MicroK8s is the smallest, fastest, fully-conformant Kubernetes that tracks upstream releases and makes clustering trivial. MicroK8s is great for offline development, prototyping, and testing.
Create a MicroK8s template
echo '#cloud-config
runcmd:
- apt update --fix-missing
- snap refresh
- snap install microk8s --classic
- microk8s status --wait-ready
- microk8s enable dns ingress' > cloud-config-microk8s.yaml
Create a MicroK8s instance
multipass launch focal -n microk8s -c 2 -m 2G -d 10G --cloud-init cloud-config-microk8s.yaml
Export the current MicroK8s configuration for use with Kubectl
Create the folder to store the configuration of the Kubernetes cluster in the test instance
sudo mkdir -p $HOME/.kube/configs
Export the MicroK8s configuration in the test instance to the created folder
multipass exec microk8s sudo microk8s config > $HOME/.kube/configs/config-microk8s
Use in your session the configuration exported as default for use with Kubectl
export KUBECONFIG=$HOME/.kube/configs/config-microk8s
Install Kubectl
sudo snap install kubectl --classic
kubectl get no
NAME STATUS ROLES AGE VERSION
microk8s Ready <none> 1m v1.21.3-3+90fd5f3d2aea0a
Add an IP alias of the test instance to teste.info
multipass info microk8s | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed 's/$/ microk8s.info/' | sudo tee -a /etc/hosts
multipass info microk8s | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed 's/$/ teste.info/' | sudo tee -a /etc/hosts
Deploy a test application
kubectl apply -f https://raw.githubusercontent.com/smashse/playbook/master/HOWTO/KUBERNETES/COMBO/example_combo_full.yaml
Grafana
Create a Grafana template
echo '#cloud-config
runcmd:
- echo "deb https://packages.grafana.com/oss/deb stable main" > /etc/apt/sources.list.d/grafana.list
- wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
- cho "DEBIAN_FRONTEND=noninteractive" >> /etc/environment
- source /etc/environment && source /etc/environment
- apt update --fix-missing
- apt -y install grafana
- systemctl daemon-reload
- systemctl start grafana-server
- systemctl status grafana-server
- systemctl enable grafana-server.service' > cloud-config-grafana.yaml
Create a Grafana instance
multipass launch focal -n grafana -c 2 -m 1G -d 10G --cloud-init cloud-config-grafana.yaml
Add an IP alias of the test instance to grafana.info
multipass info grafana | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed 's/$/ grafana.info/' | sudo tee -a /etc/hosts
Add a Data Source
To add the Data Source to Prometheus with the URL "http://prometheus.info:9090", just follow the steps as per the documentation below:
https://grafana.com/docs/grafana/latest/datasources/add-a-data-source/
Prometheus
Create a Prometheus template
echo '#cloud-config
runcmd:
- echo "deb https://packages.grafana.com/oss/deb stable main" > /etc/apt/sources.list.d/grafana.list
- wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
- cho "DEBIAN_FRONTEND=noninteractive" >> /etc/environment
- source /etc/environment && source /etc/environment
- apt update --fix-missing
- apt -y install prometheus
- systemctl daemon-reload
- systemctl start prometheus
- systemctl status prometheus
- systemctl enable prometheus.service' > cloud-config-prometheus.yaml
Create a Prometheus instance
multipass launch focal -n prometheus -c 2 -m 1G -d 10G --cloud-init cloud-config-prometheus.yaml
Add an IP alias of the test instance to prometheus.info
multipass info prometheus | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed 's/$/ prometheus.info/' | sudo tee -a /etc/hosts
Deploy the Metrics Server
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
OR
wget -c https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
sed -i '/- --secure-port=443/a\ - --kubelet-insecure-tls' components.yaml
kubectl apply -f components.yaml
kubectl get deployment metrics-server -n kube-system
Deploy the Kube-State-Metrics
kubectl apply -f https://github.com/kubernetes/kube-state-metrics/tree/master/examples/standard --recursive
OR
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install kube-state-metrics prometheus-community/kube-state-metrics --namespace monitoring --create-namespace
Istio + Basic Prometheus
Istio provides a basic sample installation to quickly get Prometheus up and running:
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.11/samples/addons/prometheus.yaml
echo 'apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: prometheus-ingress
namespace: istio-system
spec:
rules:
- host: prometheus.istio.info
http:
paths:
- backend:
service:
name: prometheus
port:
number: 9090
path: /
pathType: Prefix' > prometheus_ingress.yaml
kubectl apply -f prometheus_ingress.yaml
multipass info microk8s | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed 's/$/ prometheus.istio.info/' | sudo tee -a /etc/hosts
Allow a Prometheus server(prometheus.info) to scrape selected time series from Prometheus server running in Microk8s(prometheus.istio.info).
Access the Prometheus instance
multipass exec prometheus bash
Configuring federation
sudo echo " - job_name: 'federate'
scrape_interval: 15s
honor_labels: true
metrics_path: '/federate'
params:
'match[]':
- '{job="prometheus"}'
- '{__name__=~"job:.*"}'
static_configs:
- targets:
- 'prometheus.istio.info'" | sudo tee -a /etc/prometheus/prometheus.yml
Restart Prometheus
sudo systemctl restart prometheus
Create test deployments
echo 'africa
america
asia
europa
oceania' > lista
wget -c https://raw.githubusercontent.com/smashse/playbook/master/HOWTO/KUBERNETES/COMBO/example_combo_full.yaml
for i in `cat lista.txt` ; do sed "s/teste/$i/" example_combo_full.yaml > app-$i.yaml ; kubectl apply -f app-$i.yaml ; done
for i in `cat lista.txt` ; do multipass info microk8s | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed "s/$/ $i.info/" | sudo tee -a /etc/hosts ; done
Sources:
https://istio.io/latest/docs/ops/integrations/prometheus/
https://github.com/kubernetes-sigs/metrics-server
https://github.com/kubernetes/kube-state-metrics
https://prometheus.io/docs/prometheus/latest/federation/
https://grafana.com/grafana/dashboards/10000
https://grafana.com/grafana/dashboards/10856
Top comments (1)
Original post: linkedin.com/posts/andersongama_mu...