DEV Community: Anderson Gama

🚀 Setting Up a Kubernetes Observability Proof of Concept (POC) with MicroK8s and DataDog

Anderson Gama — Mon, 24 Feb 2025 16:09:48 +0000

Proof of Concept (POC) Requirements

In this POC, we will use Multipass to create instances for installing DataDog and MicroK8s. The goal is to establish DataDog as an observability tool for the Kubernetes cluster provisioned with MicroK8s.

1. Install Multipass

To install Multipass, run the following command:

sudo snap install multipass --classic

2. MicroK8s Setup

MicroK8s is the smallest, fastest, fully-conformant Kubernetes distribution that tracks upstream releases and simplifies clustering. It is ideal for offline development, prototyping, and testing.

2.1 Create a MicroK8s Template

Create a cloud-config-microk8s.yaml file to configure the MicroK8s instance:

echo '#cloud-config
runcmd:
  - apt update --fix-missing
  - snap refresh
  - snap install microk8s --classic
  - microk8s status --wait-ready
  - microk8s enable dns ingress' > cloud-config-microk8s.yaml

2.2 Create a MicroK8s Instance

Launch a MicroK8s instance using Multipass:

multipass launch noble -n microk8s -c 2 -m 4G -d 10G --cloud-init cloud-config-microk8s.yaml

2.3 Export MicroK8s Configuration for Kubectl

2.3.1 Create a Directory for Kubernetes Configurations

mkdir -p $HOME/.kube/configs

2.3.2 Export MicroK8s Configuration

Export the MicroK8s configuration to the created directory:

multipass exec microk8s -- sudo microk8s config | tee $HOME/.kube/configs/config-microk8s

2.3.3 Set the Exported Configuration as Default for Kubectl

export KUBECONFIG=$HOME/.kube/configs/config-microk8s
chmod 0600 $KUBECONFIG

2.3.4 Install Kubectl

Install Kubectl using Snap:

sudo snap install kubectl --classic

Verify the installation:

kubectl get no

2.3.5 Install Helm

sudo snap install helm --classic

Output:

NAME       STATUS   ROLES    AGE   VERSION
microk8s   Ready    <none>   10m   v1.32.1

2.4 Add an IP Alias for MicroK8s

Add an IP alias for the MicroK8s instance to microk8s.local:

multipass info microk8s | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed 's/$/     microk8s.local/' | sudo tee -a /etc/hosts

2.5 Access MicroK8s via URL

You can access MicroK8s via the following URL:

http://microk8s.local

3. DataDog Setup

Differences Between US1, US3, and US5 in Datadog

In Datadog, US1, US3, and US5 refer to different regions (endpoints) where data is processed and stored. These regions are designed to serve customers in specific geographic locations or to meet compliance requirements. Below are the main differences:

US1 (US Region - Default)

Endpoint: https://datadoghq.com
Description: The primary and default Datadog region for customers in the United States.
Data Residency: Data is stored and processed in the United States.
Use Case: Ideal for customers who do not have specific data residency requirements outside the US.
Performance: Best performance for users and applications located in the US.

US3 (US3 Region)

Endpoint: https://us3.datadoghq.com
Description: A separate Datadog region, also located in the United States, but isolated from US1.
Data Residency: Data is stored and processed in the United States, but in a different infrastructure than US1.
Use Case: Designed for customers who need isolation from the primary US1 region, often for compliance, regulatory, or organizational reasons.
Performance: Similar to US1 for users and applications in the US.

US5 (US5 Region)

Endpoint: https://us5.datadoghq.com
Description: Another isolated Datadog region in the United States, distinct from US1 and US3.
Data Residency: Data is stored and processed in the United States, but in a separate infrastructure.
Use Case: Used by customers who require additional isolation or have specific compliance needs that cannot be met by US1 or US3.
Performance: Comparable to US1 and US3 for users and applications in the US.

Key Differences:

Feature	US1	US3	US5
Endpoint	`datadoghq.com`	`us3.datadoghq.com`	`us5.datadoghq.com`
Location	US (Primary)	US (Isolated)	US (Isolated)
Use Case	Default US region	Compliance/isolation needs	Additional isolation needs
Performance	Optimized for US users	Optimized for US users	Optimized for US users

How to Choose the Right Region:

US1: Use this region if you do not have specific compliance or isolation requirements.
US3/US5: Use these regions if you need isolated infrastructure to meet compliance, regulatory, or organizational requirements.

Note:

Data and configurations are not shared between these regions. If you switch regions, you will need to set up your Datadog environment (dashboards, monitors, etc.) separately.
Ensure that data ingestion and API calls are directed to the correct endpoint for your chosen region.

Getting Started with Datadog Sites

3.1 Getting Started with DataDog Free Tier

Visit Datadog's website.
Create a free account.
Follow the instructions to install the Datadog Agent in your environment (on-premises or in the cloud).

3.2 Install the Datadog Agent

The Datadog Agent collects metrics and events from your systems and apps. Install at least one Agent anywhere, even on your workstation.

3.2.1 Installing on Kubernetes

Option 1: Using the Datadog Operator

The Datadog Operator is a way to deploy the Datadog Agent on Kubernetes and OpenShift. It simplifies deployment and reduces the risk of misconfiguration.

3.2.1.1 Install the Datadog Operator

Install the Datadog Operator in the current namespace:

helm repo add datadog https://helm.datadoghq.com
helm install datadog-operator datadog/datadog-operator
kubectl create secret generic datadog-secret --from-literal api-key="YOUR_API-KEY"

3.2.1.2 Configure `datadog-agent.yaml`

To configure the Datadog Agent on AKS, Openshift, and TKG distributions refer to Kubernetes distributions documentation.

Configure the Datadog Agent for your Kubernetes distribution:

echo 'apiVersion: "datadoghq.com/v2alpha1"
kind: "DatadogAgent"
metadata:
  name: "datadog"
spec:
  global:
    clusterName: "microk8s"
    site: "us5.datadoghq.com"
    credentials:
      apiSecret:
        secretName: "datadog-secret"
        keyName: "api-key"
  features:
    apm:
      instrumentation:
        enabled: true
        libVersions:
          java: "1"
          dotnet: "3"
          python: "2"
          js: "5"
    logCollection:
      enabled: true
      containerCollectAll: true
    asm:
      threats:
        enabled: true
      sca:
        enabled: true
      iast:
        enabled: true
    cws:
      enabled: true
    usm:
      enabled: true
    npm:
      enabled: true' | tee datadog-agent.yaml

3.2.1.3 Deploy the Datadog Agent

Apply the configuration:

kubectl apply -f datadog-agent.yaml

3.2.1.4 Confirm Agent Installation

Query the pods to confirm the installation:

kubectl get pods -l app.kubernetes.io/component=agent
kubectl get pods -l app.kubernetes.io/managed-by=datadog-operator

Option 2: Using the Helm Chart

You can use the Datadog Helm chart to install the Datadog Agent across all nodes in your cluster via a DaemonSet.

3.2.2.1 Add the Datadog Helm Repository

helm repo add datadog https://helm.datadoghq.com
helm repo update
kubectl create secret generic datadog-secret --from-literal api-key="YOUR_API-KEY"

3.2.2.2 Configure `datadog-values.yaml`

To configure the Datadog Agent on AKS, Openshift, and TKG distributions refer to Kubernetes distributions documentation.

Configure the Datadog Agent for your Kubernetes distribution:

echo 'datadog:
  apiKeyExistingSecret: "datadog-secret"
  site: "us5.datadoghq.com"
  apm:
    instrumentation:
      enabled: true
      libVersions:
        java: "1"
        dotnet: "3"
        python: "2"
        js: "5"
  logs:
    enabled: true
    containerCollectAll: true
  asm:
    threats:
      enabled: true
    sca:
      enabled: true
    iast:
      enabled: true
  securityAgent:
    runtime:
      enabled: true
  serviceMonitoring:
    enabled: true
  networkMonitoring:
    enabled: true' | tee datadog-values.yaml

3.2.2.3 Deploy the Datadog Agent

Install the Datadog Agent using Helm:

helm install datadog-agent -f datadog-values.yaml datadog/datadog

3.2.2.4 Confirm Agent Installation

Query the pods to confirm the installation:

kubectl get pods -l app.kubernetes.io/component=agent
kubectl get pods -l app.kubernetes.io/managed-by=Helm

Source: https://github.com/smashse/playbook/blob/master/HOWTO/DATADOG/

Multipass + Cloud-Init

Anderson Gama — Mon, 22 Jul 2024 19:49:28 +0000

Multipass

Multipass is a mini-cloud on your workstation using native hypervisors of all the supported plaforms (Windows, macOS and Linux). Multipass can launch and run virtual machines and configure them with cloud-init like a public cloud.

Install Multipass

sudo snap install multipass --classic

Debian in Multipass

echo '#cloud-config
write_files:
  - path: /etc/bash.bashrc
    content: |
      #powerline
      if [ -f /usr/bin/powerline-daemon ]; then
        powerline-daemon --quiet
        POWERLINE_BASH_CONTINUATION=1
        POWERLINE_BASH_SELECT=1
        . /usr/share/powerline/bindings/bash/powerline.sh
      fi
    append: true
runcmd:
 - apt install -y powerline' > cloud-config-debian.yaml

mkdir -p $HOME/Multipass/Debian
wget -c https://cloud.debian.org/images/cloud/bullseye/daily/latest/debian-11-genericcloud-amd64-daily.qcow2 -O $HOME/Multipass/Debian/debian.qcow2
multipass launch --name debian file://$HOME/Multipass/Debian/debian.qcow2 --cloud-init cloud-config-debian.yaml
multipass list
multipass sh debian

Rocky in Multipass

echo '#cloud-config
write_files:
  - path: /etc/dnf/dnf.conf
    content: |
      #fastest
      max_parallel_downloads=10
      fastestmirror=True
    append: true
write_files:
  - path: /etc/bashrc
    content: |
      #ohmyposh
      eval "$(oh-my-posh init bash --config /opt/themes/paradox.omp.json)"
    append: true
runcmd:
 - dnf config-manager --set-enabled crb
 - dnf install -y curl git wget zip
 - curl -s https://ohmyposh.dev/install.sh | bash -s -- -d /bin
 - mkdir -p /opt/themes
 - wget -c https://raw.githubusercontent.com/JanDeDobbeleer/oh-my-posh/main/themes/paradox.omp.json -O /opt/themes/paradox.omp.json' > cloud-config-rocky.yaml

mkdir -p $HOME/Multipass/Rocky
wget -c http://dl.rockylinux.org/pub/rocky/9/images/x86_64/Rocky-9-GenericCloud.latest.x86_64.qcow2 -O $HOME/Multipass/Rocky/rocky.qcow2
multipass launch --name rocky file://$HOME/Multipass/Rocky/rocky.qcow2 --cloud-init cloud-config-rocky.yaml
multipass list
multipass sh rocky

Matriz de Roles y Responsabilidades

Anderson Gama — Tue, 23 Apr 2024 00:02:32 +0000

La conurbación de roles en escuadrones de desarrollo ágil representa la intersección e interdependencia de las Funciones técnicas, como los SRE, los Desarrolladores Frontend y Backend. Esto implica desafíos específicos para los líderes técnicos, que deben navegar en un ambiente donde las fronteras entre las responsabilidades son fluidas y las competencias se superponen. Los desafíos incluyen:

Definición de Responsabilidades: La demarcación clara de responsabilidades es compleja cuando los roles comparten tareas, como la optimización del rendimiento, que puede ser una zona gris entre los SRE y los Desarrolladores Backend.

Desarrollo de Competencias Técnicas: La evolución tecnológica exige que los profesionales tengan competencias que trasciendan sus especializaciones tradicionales, como los Desarrolladores Frontend con conocimientos en Backend.

Comunicación Efectiva: La estrecha colaboración entre funciones distintas exige una comunicación efectiva para evitar malentendidos y garantizar que todos los miembros del escuadrón estén alineados con los objetivos comunes.

Gestión de Límites Funcionales: Establecer límites claros entre las funciones es esencial para prevenir redundancias y lagunas en las responsabilidades, lo que requiere un liderazgo técnico asertivo y consciente.

Cultura de Colaboración: Promover una cultura de trabajo que valore la colaboración, el aprendizaje continuo y el intercambio de conocimientos es fundamental en un modelo de escuadrón menos jerárquico.

Adaptación a Cambios Tecnológicos: Los líderes técnicos deben estar preparados para guiar a sus equipos a través de cambios tecnológicos rápidos y frecuentes, lo que exige flexibilidad y una mentalidad ágil.

¿Cuál es el papel de un SRE en un escuadrón?

Un SRE (Ingeniero de Confiabilidad del Sitio) en un escuadrón juega un papel crucial en garantizar la confiabilidad y estabilidad de los sistemas. Aquí están algunas de las responsabilidades e interacciones típicas de un SRE en un escuadrón:

Responsabilidades: El SRE es responsable de garantizar la confiabilidad y estabilidad de los sistemas que el escuadrón está desarrollando y manteniendo. Esto incluye el desarrollo de soluciones de automatización para tareas repetitivas, monitoreo de los sistemas para identificar y responder a incidentes, y planificación de capacidad para garantizar que los sistemas puedan manejar la demanda prevista.

Colaboración: El SRE trabaja en estrecha colaboración con otros miembros del escuadrón, que pueden incluir desarrolladores, diseñadores de UX, y otros. Pueden interactuar con arquitectos de sistemas para garantizar que los sistemas que se están desarrollando sean escalables y puedan manejar la carga prevista.

Relación con Otros Roles: El SRE actúa como un puente entre los equipos de desarrollo y operaciones, contribuyendo a la cultura de ingeniería de confiabilidad. Ayudan a garantizar que las prácticas de desarrollo y operaciones estén alineadas con los objetivos de confiabilidad y estabilidad del sistema.

¿Cuál es el papel de un Desarrollador Frontend en un escuadrón?

Un Desarrollador Frontend en un escuadrón tiene un papel crucial en la creación y manutenimiento de la interfaz de usuario de una aplicación o sitio web. Aquí están algunas de las responsabilidades e interacciones típicas de un Desarrollador Frontend en un escuadrón:

Responsabilidades: El Desarrollador Frontend es responsable de desarrollar una interfaz que permita al usuario lograr sus objetivos de manera intuitiva. Esto incluye la creación de la parte visual e interactiva de los sitios web y aplicaciones, la optimización de la experiencia del usuario, la garantía de responsividad y compatibilidad entre dispositivos y navegadores, y la implementación de SEO.

Colaboración: El Desarrollador Frontend trabaja en estrecha colaboración con otros miembros del escuadrón, que pueden incluir diseñadores de UI/UX, desarrolladores backend, y otros. Colaboran con los diseñadores de UI/UX para traducir visuales en código y trabajan junto a los desarrolladores backend para integrar APIs y servicios.

Relación con Otros Roles: El Desarrollador Frontend actúa como el puente entre el diseño gráfico y la programación. Impactan directamente en la experiencia del usuario final. Además, también pueden interactuar con el Líder Técnico (Tech Lead), que guía al escuadrón, gestiona el proyecto, garantiza una comunicación fluida, controla el pipeline del proyecto y garantiza las entregas, gestiona el equipo técnico, manteniendo un ambiente de trabajo saludable, investiga nuevas tecnologías y comparte la cultura de la innovación, y asume un papel de mentor, ayudando individualmente a los miembros.

¿Cuál es el papel de un Desarrollador Backend en un escuadrón?

Un Desarrollador Backend en un escuadrón tiene un papel crucial en la creación y manutenimiento de la lógica del servidor y las funcionalidades de los bastidores de una aplicación o sitio web. Aquí están algunas de las responsabilidades e interacciones típicas de un Desarrollador Backend en un escuadrón:

Responsabilidades: El Desarrollador Backend es responsable de desarrollar, probar y mantener el código que controla el funcionamiento del servidor, la base de datos y otras funcionalidades de los bastidores. Esto incluye la creación e implementación de las reglas de negocio y funcionalidades de un sistema digital, la modelación y gestión de bases de datos relacionales y no relacionales, el desarrollo de APIs que permiten la integración entre front-end y back-end, y la garantía de buen rendimiento, escalabilidad y seguridad de la aplicación.

Colaboración: El Desarrollador Backend trabaja en estrecha colaboración con otros miembros del escuadrón, que pueden incluir desarrolladores frontend, diseñadores de UI/UX, y otros. Colaboran con los desarrolladores frontend para garantizar que la lógica del servidor soporte la interfaz del usuario.

Relación con Otros Roles: El Desarrollador Backend es responsable de la funcionalidad ‘invisível’ que soporta el frontend. Desempeñan un papel crucial en la seguridad e integridad de los datos. Además, también pueden interactuar con el Líder Técnico (Tech Lead), que guía al escuadrón, gestiona el proyecto, garantiza una comunicación fluida, controla el pipeline del proyecto y garantiza las entregas, gestiona el equipo técnico, manteniendo un ambiente de trabajo saludable, investiga nuevas tecnologías y comparte la cultura de la innovación, y asume un papel de mentor, ayudando individualmente a los miembros.

Criterio	SRE	Desarrollador Frontend	Desarrollador Backend
Responsabilidades	- Garantizar la confiabilidad y estabilidad de los sistemas. - Desarrollar soluciones de automatización. - Monitorear sistemas y responder a incidentes. - Planificación de capacidad y gestión de riesgos.	- Crear la parte visual e interactiva de sitios web y aplicaciones. - Optimizar la experiencia del usuario. - Asegurar la responsividad y compatibilidad entre dispositivos y navegadores. - Implementar SEO.	- Desarrollar y mantener el lado del servidor de las aplicaciones. - Gestionar bases de datos y lógica de negocios. - Asegurar la seguridad y rendimiento del sistema. - Integrar APIs y servicios externos.
Habilidades Técnicas	- Conocimiento en automatización y herramientas de CI/CD. - Habilidades en monitoreo y alertas. - Experiencia con lenguajes de programación y scripts. - Conocimiento en redes y sistemas operativos.	- Dominio de HTML, CSS y JavaScript. - Experiencia con frameworks como React o Angular. - Conocimiento en diseño responsivo y cross-browser. - Familiaridad con herramientas de diseño como Adobe XD o Sketch.	- Proficiencia en lenguajes de programación como Java, Python, Ruby o PHP. - Experiencia con frameworks de backend. - Conocimiento en bases de datos SQL y NoSQL. - Habilidades en crear y gestionar APIs.
Colaboración	- Trabaja en estrecha colaboración con equipos de desarrollo y operaciones. - Interactúa con arquitectos de sistemas para garantizar la escalabilidad.	- Colabora con diseñadores de UI/UX para traducir visuales en código. - Trabaja junto a desarrolladores backend para integrar APIs y servicios.	- Colabora con desarrolladores frontend para garantizar que la lógica del servidor soporte la interfaz del usuario. - Trabaja con equipos de bases de datos y seguridad.
Relación con Otros Roles	- Actúa como puente entre operaciones y desarrollo (DevOps). - Contribuye a la cultura de ingeniería de confiabilidad.	- Es el puente entre el diseño gráfico y la programación. - Impacta directamente en la experiencia del usuario final.	- Responsable por la funcionalidad 'invisible' que soporta el frontend. - Desempeña un papel crucial en la seguridad e integridad de los datos.
Relación entre los Roles	- Colabora con Desarrolladores Frontend y Backend para garantizar la confiabilidad y estabilidad de los sistemas. - Proporciona retroalimentación y orientación para optimizar el rendimiento y la escalabilidad.	- Trabaja en conjunto con el Desarrollador Backend para integrar la interfaz del usuario con la lógica del servidor. - Colabora con el SRE para garantizar que la interfaz del usuario sea estable y confiable.	- Trabaja en conjunto con el Desarrollador Frontend para proporcionar una API estable y eficiente. - Colabora con el SRE para garantizar que la lógica del servidor sea confiable y escalable.

Roles and Responsibilities Matrix

Anderson Gama — Sun, 21 Apr 2024 04:21:55 +0000

The conurbation of roles in agile development squads represents the intersection and interdependence of technical functions, such as SREs, Frontend and Backend Developers. This implies specific challenges for technical leaders, who must navigate an environment where the boundaries between responsibilities are fluid and competencies overlap. The challenges include:

Responsibility Definition: Clear demarcation of responsibilities is complex when roles share tasks, such as performance optimization, which can be a gray area between SREs and Backend Developers.

Technical Competency Development: Technological evolution requires professionals to have competencies that transcend their traditional specializations, such as Frontend Developers with knowledge in Backend.

Effective Communication: Close collaboration between distinct functions demands effective communication to avoid misunderstandings and ensure that all squad members are aligned with common goals.

Functional Boundaries Management: Establishing clear boundaries between roles is essential to prevent redundancies and gaps in responsibilities, which requires assertive and conscious technical leadership.

Collaboration Culture: Promoting a work culture that values collaboration, continuous learning, and knowledge sharing is fundamental in a less hierarchical squad model.

Adaptation to Technological Changes: Technical leaders must be prepared to guide their teams through rapid and frequent technological changes, which requires flexibility and an agile mindset.

What is the role of an SRE in a squad?

An SRE (Site Reliability Engineer) in a squad plays a crucial role in ensuring the reliability and stability of systems. Here are some of the typical responsibilities and interactions of an SRE in a squad:

Responsibilities: The SRE is responsible for ensuring the reliability and stability of the systems that the squad is developing and maintaining. This includes the development of automation solutions for repetitive tasks, monitoring systems to identify and respond to incidents, and capacity planning to ensure that systems can handle the anticipated demand.

Collaboration: The SRE works closely with other squad members, which may include developers, UX designers, and others. They may interact with system architects to ensure that the systems being developed are scalable and can handle the anticipated load.

Relationship with Other Roles: The SRE acts as a bridge between the development and operations teams, contributing to the culture of reliability engineering. They help ensure that development and operations practices are aligned with the system's reliability and stability objectives.

What is the role of a Frontend Developer in a squad?

A Frontend Developer in a squad plays a crucial role in creating and maintaining the user interface of an application or website. Here are some of the typical responsibilities and interactions of a Frontend Developer in a squad:

Responsibilities: The Frontend Developer is responsible for developing an interface that allows the user to achieve their goals intuitively. This includes creating the visual and interactive part of websites and applications, optimizing the user experience, ensuring responsiveness and compatibility across devices and browsers, and implementing SEO.

Collaboration: The Frontend Developer works closely with other squad members, which may include UI/UX designers, backend developers, and others. They collaborate with UI/UX designers to translate visuals into code and work alongside backend developers to integrate APIs and services.

Relationship with Other Roles: The Frontend Developer acts as the bridge between graphic design and programming. They directly impact the end user experience. In addition, they may also interact with the Technical Leader (Tech Lead), who guides the squad, manages the project, ensures fluid communication, controls the project pipeline and ensures deliveries, manages the technical team, maintaining a healthy work environment, researches new technologies and shares the culture of innovation, and takes on a mentoring role, helping individual members individually.

What is the role of a Backend Developer in a squad?

A Backend Developer in a squad plays a crucial role in creating and maintaining the server logic and backstage functionalities of an application or website. Here are some of the typical responsibilities and interactions of a Backend Developer in a squad:

Responsibilities: The Backend Developer is responsible for developing, testing, and maintaining the code that controls the operation of the server, database, and other backstage functionalities. This includes creating and implementing the business rules and functionalities of a digital system, modeling and managing relational and non-relational databases, developing APIs that allow integration between front-end and back-end, and ensuring good performance, scalability, and security of the application.

Collaboration: The Backend Developer works closely with other squad members, which may include frontend developers, UI/UX designers, and others. They collaborate with frontend developers to ensure that the server logic supports the user interface.

Relationship with Other Roles: The Backend Developer is responsible for the 'invisible' functionality that supports the frontend. They play a crucial role in data security and integrity. In addition, they may also interact with the Technical Leader (Tech Lead), who guides the squad, manages the project, ensures fluid communication, controls the project pipeline and ensures deliveries, manages the technical team, maintaining a healthy work environment, researches new technologies and shares the culture of innovation, and takes on a mentoring role, helping individual members.

Criterion	SRE	Frontend Developer	Backend Developer
Responsibilities	- Ensure the reliability and stability of systems. - Develop automation solutions. - Monitor systems and respond to incidents. - Capacity planning and risk management.	- Create the visual and interactive part of websites and applications. - Optimize the user experience. - Ensure responsiveness and compatibility across devices and browsers. - Implement SEO.	- Develop and maintain the server side of applications. - Manage databases and business logic. - Ensure system security and performance. - Integrate APIs and external services.
Technical Skills	- Knowledge in automation and CI/CD tools. - Skills in monitoring and alerts. - Experience with programming languages and scripts. - Knowledge in networks and operating systems.	- Mastery of HTML, CSS and JavaScript. - Experience with frameworks like React or Angular. - Knowledge in responsive and cross-browser design. - Familiarity with design tools like Adobe XD or Sketch.	- Proficiency in programming languages like Java, Python, Ruby or PHP. - Experience with backend frameworks. - Knowledge in SQL and NoSQL databases. - Skills in creating and managing APIs.
Collaboration	- Works closely with development and operations teams. - Interacts with system architects to ensure scalability.	- Collaborates with UI/UX designers to translate visuals into code. - Works alongside backend developers to integrate APIs and services.	- Collaborates with frontend developers to ensure that the server logic supports the user interface. - Works with database and security teams.
Relationship with Other Roles	- Acts as a bridge between operations and development (DevOps). - Contributes to the culture of reliability engineering.	- Is the bridge between graphic design and programming. - Directly impacts the end user experience.	- Responsible for the 'invisible' functionality that supports the frontend. - Plays a crucial role in data security and integrity.
Relationship between Roles	- Collaborates with Frontend and Backend Developers to ensure the reliability and stability of systems. - Provides feedback and guidance to optimize performance and scalability.	- Works together with the Backend Developer to integrate the user interface with the server logic. - Collaborates with the SRE to ensure that the user interface is stable and reliable.	- Works together with the Frontend Developer to provide a stable and efficient API. - Collaborates with the SRE to ensure that the server logic is reliable and scalable.

Matriz de Papéis e Responsabilidades

Anderson Gama — Sat, 20 Apr 2024 21:50:34 +0000

A conurbação de papéis em squads de desenvolvimento ágil representa a interseção e interdependência das funções técnicas, como SRE´s, Desenvolvedores Frontend e Backend. Isso implica desafios específicos para líderes técnicos, que devem navegar por um ambiente onde as fronteiras entre as responsabilidades são fluidas e as competências se sobrepõem. Os desafios incluem:

Definição de Responsabilidades: A demarcação clara de responsabilidades é complexa quando as funções compartilham tarefas, como a otimização de desempenho, que pode ser uma área cinzenta entre SRE´s e Desenvolvedores Backend.

Desenvolvimento de Competências Técnicas: A evolução tecnológica exige que os profissionais tenham competências que transcendam suas especializações tradicionais, como Desenvolvedores Frontend com conhecimento em Backend.

Comunicação Efetiva: A colaboração estreita entre funções distintas demanda uma comunicação eficaz para evitar mal-entendidos e garantir que todos os membros da squad estejam alinhados com os objetivos comuns.

Gestão de Limites Funcionais: Estabelecer limites claros entre as funções é essencial para prevenir redundâncias e lacunas nas responsabilidades, o que requer uma liderança técnica assertiva e consciente.

Cultura de Colaboração: Promover uma cultura de trabalho que valorize a colaboração, a aprendizagem contínua e o compartilhamento de conhecimento é fundamental em um modelo de squad menos hierárquico.

Adaptação a Mudanças Tecnológicas: Líderes técnicos devem estar preparados para orientar suas equipes através de mudanças tecnológicas rápidas e frequentes, o que exige flexibilidade e uma mentalidade ágil.

Qual é o papel de um SRE em uma squad?

Um SRE (Site Reliability Engineer) em uma squad desempenha um papel crucial na garantia da confiabilidade e estabilidade dos sistemas. Aqui estão algumas das responsabilidades e interações típicas de um SRE em uma squad:

Responsabilidades: O SRE é responsável por garantir a confiabilidade e estabilidade dos sistemas que a squad está desenvolvendo e mantendo. Isso inclui o desenvolvimento de soluções de automação para tarefas repetitivas, monitoramento dos sistemas para identificar e responder a incidentes, e planejamento de capacidade para garantir que os sistemas possam lidar com a demanda prevista.

Colaboração: O SRE trabalha em estreita colaboração com outros membros da squad, que podem incluir desenvolvedores, designers de UX, e outros. Eles podem interagir com arquitetos de sistemas para garantir que os sistemas que estão sendo desenvolvidos sejam escaláveis e possam lidar com a carga prevista.

Relação com Outros Papéis: O SRE atua como uma ponte entre as equipes de desenvolvimento e operações, contribuindo para a cultura de engenharia de confiabilidade. Eles ajudam a garantir que as práticas de desenvolvimento e operações estejam alinhadas com os objetivos de confiabilidade e estabilidade do sistema.

Qual é o papel de um Desenvolvedor Frontend em uma squad?

Um Desenvolvedor Frontend em uma squad tem um papel crucial na criação e manutenção da interface do usuário de um aplicativo ou site. Aqui estão algumas das responsabilidades e interações típicas de um Desenvolvedor Frontend em uma squad:

Responsabilidades: O Desenvolvedor Frontend é responsável por desenvolver uma interface que permita que o usuário consiga realizar seus objetivos de forma intuitiva. Isso inclui a criação da parte visual e interativa de websites e aplicativos, a otimização da experiência do usuário, a garantia de responsividade e compatibilidade entre dispositivos e navegadores, e a implementação de SEO.

Colaboração: O Desenvolvedor Frontend trabalha em estreita colaboração com outros membros da squad, que podem incluir designers de UI/UX, desenvolvedores backend, e outros. Eles colaboram com designers de UI/UX para traduzir visuais em código e trabalham junto a desenvolvedores backend para integrar APIs e serviços.

Relação com Outros Papéis: O Desenvolvedor Frontend atua como a ponte entre o design gráfico e a programação. Eles impactam diretamente a experiência do usuário final. Além disso, eles também podem interagir com o Líder Técnico (Tech Lead), que orienta o squad, gerencia o projeto, garante uma comunicação fluída, controla o pipeline do projeto e garante as entregas, gerencia a equipe técnica, mantendo um ambiente de trabalho saudável, pesquisa novas tecnologias e compartilha a cultura da inovação, e assume um papel de mentor, ajudando individualmente os membros.

Qual é o papel de um Desenvolvedor Backend em uma squad?

Um Desenvolvedor Backend em uma squad tem um papel crucial na criação e manutenção da lógica do servidor e das funcionalidades dos bastidores de um aplicativo ou site. Aqui estão algumas das responsabilidades e interações típicas de um Desenvolvedor Backend em uma squad:

Responsabilidades: O Desenvolvedor Backend é responsável por desenvolver, testar e manter o código que controla o funcionamento do servidor, banco de dados e outras funcionalidades dos bastidores. Isso inclui a criação e implementação das regras de negócio e funcionalidades de um sistema digital, a modelagem e gerenciamento de bancos de dados relacionais e não-relacionais, o desenvolvimento de APIs que permitem a integração entre front-end e back-end, e a garantia de boa performance, escalabilidade e segurança da aplicação.

Colaboração: O Desenvolvedor Backend trabalha em estreita colaboração com outros membros da squad, que podem incluir desenvolvedores frontend, designers de UI/UX, e outros. Eles colaboram com desenvolvedores frontend para garantir que a lógica do servidor suporte a interface do usuário.

Relação com Outros Papéis: O Desenvolvedor Backend é responsável pela funcionalidade ‘invisível’ que suporta o frontend. Eles desempenham um papel crucial na segurança e integridade dos dados. Além disso, eles também podem interagir com o Líder Técnico (Tech Lead), que orienta o squad, gerencia o projeto, garante uma comunicação fluída, controla o pipeline do projeto e garante as entregas, gerencia a equipe técnica, mantendo um ambiente de trabalho saudável, pesquisa novas tecnologias e compartilha a cultura da inovação, e assume um papel de mentor, ajudando individualmente os membros.

Critério	SRE	Desenvolvedor Frontend	Desenvolvedor Backend
Responsabilidades	- Garantir a confiabilidade e estabilidade dos sistemas. - Desenvolver soluções de automação. - Monitorar sistemas e responder a incidentes. - Planejamento de capacidade e gestão de riscos.	- Criar a parte visual e interativa de websites e aplicativos. - Otimizar a experiência do usuário. - Assegurar a responsividade e compatibilidade entre dispositivos e navegadores. - Implementar SEO.	- Desenvolver e manter o lado do servidor de aplicações. - Gerenciar bancos de dados e lógica de negócios. - Assegurar segurança e desempenho do sistema. - Integrar APIs e serviços externos.
Habilidades Técnicas	- Conhecimento em automação e ferramentas de CI/CD. - Habilidades em monitoramento e alertas. - Experiência com linguagens de programação e scripts. - Conhecimento em redes e sistemas operacionais.	- Domínio de HTML, CSS e JavaScript. - Experiência com frameworks como React ou Angular. - Conhecimento em design responsivo e cross-browser. - Familiaridade com ferramentas de design como Adobe XD ou Sketch.	- Proficiência em linguagens de programação como Java, Python, Ruby ou PHP. - Experiência com frameworks de backend. - Conhecimento em bancos de dados SQL e NoSQL. - Habilidades em criar e gerenciar APIs.
Colaboração	- Trabalha em estreita colaboração com equipes de desenvolvimento e operações. - Interage com arquitetos de sistemas para garantir a escalabilidade.	- Colabora com designers de UI/UX para traduzir visuais em código. - Trabalha junto a desenvolvedores backend para integrar APIs e serviços.	- Colabora com desenvolvedores frontend para garantir que a lógica do servidor suporte a interface do usuário. - Trabalha com equipes de banco de dados e segurança.
Relação com Outros Papéis	- Atua como ponte entre operações e desenvolvimento (DevOps). - Contribui para a cultura de engenharia de confiabilidade.	- É a ponte entre o design gráfico e a programação. - Impacta diretamente a experiência do usuário final.	- Responsável pela funcionalidade 'invisível' que suporta o frontend. - Desempenha um papel crucial na segurança e integridade dos dados.
Relação entre os Papéis	- Colabora com Desenvolvedores Frontend e Backend para garantir a confiabilidade e estabilidade dos sistemas. - Fornece feedback e orientação para otimizar o desempenho e a escalabilidade.	- Trabalha em conjunto com o Desenvolvedor Backend para integrar a interface do usuário com a lógica do servidor. - Colabora com o SRE para garantir que a interface do usuário seja estável e confiável.	- Trabalha em conjunto com o Desenvolvedor Frontend para fornecer uma API estável e eficiente. - Colabora com o SRE para garantir que a lógica do servidor seja confiável e escalável.

DevOps Environment on MacOS

Anderson Gama — Sat, 05 Feb 2022 06:12:27 +0000

Install Homebrew

Homebrew is a free and open-source package management system for Mac OS X.

https://brew.sh/

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Change default shell to bash

chsh -s /bin/bash

Install XCode

brew install xcodegen
xcode-select --install

Install Python

brew install python

Install Firefox ESR

Firefox for your enterprise.

https://www.mozilla.org/pt-BR/firefox/enterprise/

https://formulae.brew.sh/cask/firefox

brew install --cask homebrew/cask-versions/firefox-esr

Install Google Chrome

The browser built by Google.

https://www.google.com/intl/pt-BR/chrome/

https://chromeenterprise.google/browser/

https://formulae.brew.sh/cask/google-chrome

brew install --cask google-chrome

Install VSCodium

VSCodium is a community-driven, freely-licensed binary distribution of Microsoft’s editor VS Code.

https://vscodium.com/#intro

https://formulae.brew.sh/cask/vscodium

brew install --cask vscodium

Install Visual Studio Code

Visual Studio Code is a lightweight but powerful source code editor which runs on your desktop.

https://code.visualstudio.com/

https://formulae.brew.sh/cask/visual-studio-code

brew install --cask visual-studio-code

Install Kubectl

The Kubernetes command-line tool, kubectl, allows you to run commands against Kubernetes clusters.

https://kubernetes.io/docs/tasks/tools/

https://kubernetes.io/docs/tasks/tools/install-kubectl-macos/#install-with-homebrew-on-macos

brew install kubectl

Install Krew and Neat

Krew

Krew is the plugin manager for kubectl command-line tool.

https://krew.sigs.k8s.io/

https://krew.sigs.k8s.io/docs/user-guide/setup/install/

(
  set -x; cd "$(mktemp -d)" &&
  OS="$(uname | tr '[:upper:]' '[:lower:]')" &&
  ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')" &&
  KREW="krew-${OS}_${ARCH}" &&
  curl -fsSLO "https://github.com/kubernetes-sigs/krew/releases/latest/download/${KREW}.tar.gz" &&
  tar zxvf "${KREW}.tar.gz" &&
  ./"${KREW}" install krew
)

export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"

Neat

Clean up Kubernetes yaml and json output to make it readable.

https://github.com/itaysk/kubectl-neat

https://github.com/itaysk/kubectl-neat#installation

kubectl krew install neat

Install Helm

Helm is the best way to find, share, and use software built for Kubernetes.

https://helm.sh/

https://helm.sh/docs/intro/install/#from-homebrew-macos

brew install helm

Install Lens

Lens is the most powerful Kubernetes IDE on the market.

https://k8slens.dev/

brew install lens

Install Kubenav

kubenav is the navigator for your Kubernetes clusters right in your pocket. kubenav is a mobile, desktop and web app to manage Kubernetes clusters and to get an overview of the status of your resources.

https://docs.kubenav.io/

https://docs.kubenav.io/desktop/getting-started/

brew install kubenav

Install Terraform

Build, change, and destroy infrastructure with Terraform.

https://learn.hashicorp.com/terraform

https://learn.hashicorp.com/tutorials/terraform/install-cli

brew tap hashicorp/tap
brew install hashicorp/tap/terraform

Install Terraform-Docs

Generate Terraform modules documentation in various formats.

https://terraform-docs.io/

https://terraform-docs.io/user-guide/installation/#homebrew

brew tap terraform-docs/tap
brew install terraform-docs/tap/terraform-docs

Install AWScli

Official Amazon AWS command-line interface.

https://aws.amazon.com/cli/

https://formulae.brew.sh/formula/awscli

brew install awscli

Install Azure-cli

Official Azure command-line interface.

https://docs.microsoft.com/cli/azure/overview

https://formulae.brew.sh/formula/azure-cli

brew install azure-cli

Install Google Cloud SDK

Official Google Cloud command-line interface.

https://cloud.google.com/sdk/

https://formulae.brew.sh/cask/google-cloud-sdk

brew install --cask google-cloud-sdk

Install Multipass

Ubuntu VMs on demand for any workstation.

https://multipass.run/

https://multipass.run/docs/installing-on-macos

https://formulae.brew.sh/cask/multipass

brew install --cask multipass

Install Podman

Manage pods, containers, and container images.

https://podman.io/

https://podman.io/getting-started/installation

brew install podman

podman machine init
podman machine start
podman info

Install Rancher Desktop

Kubernetes and Container Management on the Desktop.

https://rancherdesktop.io/

https://iongion.github.io/podman-desktop-companion/

for i in $(curl -s https://api.github.com/repos/rancher-sandbox/rancher-desktop/releases/latest | grep 'browser_' | cut -d\" -f4 | grep "x86_64.dmg" | head -1); do wget -c $i -O "rancher_desktop.dmg"; done

cd /tmp
hdiutil attach rancher_desktop.dmg -mountpoint /Volumes/Rancher
cp -a /Volumes/Rancher/Rancher*.app /Volumes/Rancher/Applications/
umount /Volumes/Rancher

echo "
#DOCKER
alias docker='podman'" | tee -a ~/.bashrc

source ~/.bashrc

Install ITerm2

Terminal emulator as alternative to Apple's Terminal app.

https://www.iterm2.com/

https://formulae.brew.sh/cask/iterm2

brew install --cask iterm2

Install Powerline-Go

Beautiful and useful low-latency prompt for your shell.

https://github.com/justjanne/powerline-go

https://formulae.brew.sh/formula/powerline-go

brew install powerline-go

echo '
#POWERLINE
function _update_ps1() {
    PS1="$($GOPATH/bin/powerline-go -error $? -jobs $(jobs -p | wc -l))"
}

if [ "$TERM" != "linux" ] && [ -f "$GOPATH/bin/powerline-go" ]; then
    PROMPT_COMMAND="_update_ps1; $PROMPT_COMMAND"
fi' | tee -a ~/.profile

source ~/.profile

Install the PowerlineSymbols font from:

https://github.com/powerline/powerline/tree/develop/font

In ITerm2 Preferences, go to "Text", "Font" and enable "Use a different font for non-ASCII text" and then select "PowerlineSymbols" as "Non-ASCII Font".

Multipass, Microk8s, Prometheus and Grafana

Anderson Gama — Sun, 29 Aug 2021 02:37:13 +0000

Requirements for this POC

This POC will use Multipass to create instances to install Grafana, Prometheus and Microk8s, we will install Metrics Server, Kube-State-Metrics and Istio + Basic Prometheus and then we will make metrics used by our main instance with Prometheus.

Install Multipass

sudo snap install multipass --classic

MicroK8s

MicroK8s is the smallest, fastest, fully-conformant Kubernetes that tracks upstream releases and makes clustering trivial. MicroK8s is great for offline development, prototyping, and testing.

Create a MicroK8s template

echo '#cloud-config
runcmd:
 - apt update --fix-missing
 - snap refresh
 - snap install microk8s --classic
 - microk8s status --wait-ready
 - microk8s enable dns ingress' > cloud-config-microk8s.yaml

Create a MicroK8s instance

multipass launch focal -n microk8s -c 2 -m 2G -d 10G --cloud-init cloud-config-microk8s.yaml

Export the current MicroK8s configuration for use with Kubectl

Create the folder to store the configuration of the Kubernetes cluster in the test instance

sudo mkdir -p $HOME/.kube/configs

Export the MicroK8s configuration in the test instance to the created folder

multipass exec microk8s sudo microk8s config > $HOME/.kube/configs/config-microk8s

Use in your session the configuration exported as default for use with Kubectl

export KUBECONFIG=$HOME/.kube/configs/config-microk8s

Install Kubectl

sudo snap install kubectl --classic

kubectl get no

NAME       STATUS   ROLES    AGE   VERSION
microk8s   Ready    <none>   1m    v1.21.3-3+90fd5f3d2aea0a

Add an IP alias of the test instance to teste.info

multipass info microk8s | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed 's/$/     microk8s.info/' | sudo tee -a /etc/hosts
multipass info microk8s | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed 's/$/     teste.info/' | sudo tee -a /etc/hosts

Deploy a test application

kubectl apply -f https://raw.githubusercontent.com/smashse/playbook/master/HOWTO/KUBERNETES/COMBO/example_combo_full.yaml

Grafana

Create a Grafana template

echo '#cloud-config
runcmd:
 - echo "deb https://packages.grafana.com/oss/deb stable main" > /etc/apt/sources.list.d/grafana.list
 - wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
 - cho "DEBIAN_FRONTEND=noninteractive" >> /etc/environment
 - source /etc/environment && source /etc/environment
 - apt update --fix-missing
 - apt -y install grafana
 - systemctl daemon-reload
 - systemctl start grafana-server
 - systemctl status grafana-server
 - systemctl enable grafana-server.service' > cloud-config-grafana.yaml

Create a Grafana instance

multipass launch focal -n grafana -c 2 -m 1G -d 10G --cloud-init cloud-config-grafana.yaml

Add an IP alias of the test instance to grafana.info

multipass info grafana | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed 's/$/     grafana.info/' | sudo tee -a /etc/hosts

Add a Data Source

To add the Data Source to Prometheus with the URL "http://prometheus.info:9090", just follow the steps as per the documentation below:

https://grafana.com/docs/grafana/latest/datasources/add-a-data-source/

Prometheus

Create a Prometheus template

echo '#cloud-config
runcmd:
 - echo "deb https://packages.grafana.com/oss/deb stable main" > /etc/apt/sources.list.d/grafana.list
 - wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
 - cho "DEBIAN_FRONTEND=noninteractive" >> /etc/environment
 - source /etc/environment && source /etc/environment
 - apt update --fix-missing
 - apt -y install prometheus
 - systemctl daemon-reload
 - systemctl start prometheus
 - systemctl status prometheus
 - systemctl enable prometheus.service' > cloud-config-prometheus.yaml

Create a Prometheus instance

multipass launch focal -n prometheus -c 2 -m 1G -d 10G --cloud-init cloud-config-prometheus.yaml

Add an IP alias of the test instance to prometheus.info

multipass info prometheus | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed 's/$/     prometheus.info/' | sudo tee -a /etc/hosts

Deploy the Metrics Server

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

wget -c https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

sed -i '/- --secure-port=443/a\        - --kubelet-insecure-tls' components.yaml

kubectl apply -f components.yaml

kubectl get deployment metrics-server -n kube-system

Deploy the Kube-State-Metrics

kubectl apply -f https://github.com/kubernetes/kube-state-metrics/tree/master/examples/standard --recursive

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install kube-state-metrics prometheus-community/kube-state-metrics --namespace monitoring --create-namespace

Istio + Basic Prometheus

Istio provides a basic sample installation to quickly get Prometheus up and running:

kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.11/samples/addons/prometheus.yaml

echo 'apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: prometheus-ingress
  namespace: istio-system
spec:
  rules:
  - host: prometheus.istio.info
    http:
      paths:
      - backend:
          service:
            name: prometheus
            port:
              number: 9090
        path: /
        pathType: Prefix' > prometheus_ingress.yaml

kubectl apply -f prometheus_ingress.yaml

multipass info microk8s | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed 's/$/     prometheus.istio.info/' | sudo tee -a /etc/hosts

Allow a Prometheus server(prometheus.info) to scrape selected time series from Prometheus server running in Microk8s(prometheus.istio.info).

Access the Prometheus instance

multipass exec prometheus bash

Configuring federation

sudo echo "  - job_name: 'federate'
    scrape_interval: 15s

    honor_labels: true
    metrics_path: '/federate'

    params:
      'match[]':
        - '{job="prometheus"}'
        - '{__name__=~"job:.*"}'

    static_configs:
      - targets:
        - 'prometheus.istio.info'" | sudo tee -a /etc/prometheus/prometheus.yml

Restart Prometheus

sudo systemctl restart prometheus

Create test deployments

echo 'africa
america
asia
europa
oceania' > lista

wget -c https://raw.githubusercontent.com/smashse/playbook/master/HOWTO/KUBERNETES/COMBO/example_combo_full.yaml

for i in `cat lista.txt` ; do sed "s/teste/$i/" example_combo_full.yaml > app-$i.yaml ; kubectl apply -f app-$i.yaml ; done

for i in `cat lista.txt` ; do multipass info microk8s | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed "s/$/     $i.info/" | sudo tee -a /etc/hosts ; done

Sources:

https://istio.io/latest/docs/ops/integrations/prometheus/

https://github.com/kubernetes-sigs/metrics-server

https://github.com/kubernetes/kube-state-metrics

https://github.com/helm/charts/tree/master/stable/prometheus-operator/templates/grafana/dashboards-1.14

https://prometheus.io/docs/prometheus/latest/federation/

https://grafana.com/grafana/dashboards/10000

https://grafana.com/grafana/dashboards/10856

https://grafana.com/grafana/dashboards/10858

https://grafana.com/grafana/dashboards/8588

Install Powerline for Powershell

Anderson Gama — Sun, 16 May 2021 20:46:58 +0000

I always used Powerline in Linux but I didn't know it had an option for Windows, very nice :-)

Powerline provides a customized command prompt experience providing Git status color-coding and prompts.

Set execution policy

Get-ExecutionPolicy -List
Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope Process
Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope CurrentUser
Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope LocalMachine

Install Chocolatey

Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))

Install Git on Windows

choco install git wget unzip code

Install Powerline in PowerShell

Using PowerShell, install Posh-Git and Oh-My-Posh:

Install-Module posh-git -Scope CurrentUser
Install-Module oh-my-posh -Scope CurrentUser

If you are using PowerShell Core, install PSReadline:

Install-Module -Name PSReadLine -Scope CurrentUser -Force -SkipPublisherCheck

In your PowerShell profile, add the following to the end of the file and Customize your PowerShell prompt:

echo 'Import-Module posh-git
Import-Module oh-my-posh
Set-PoshPrompt -Theme paradox' > $PROFILE

List of themes: https://ohmyposh.dev/docs/themes/

Powerline fonts: https://github.com/powerline/fonts

cd ${HOME}\Downloads\
wget -c https://github.com/powerline/fonts/archive/refs/heads/master.zip
unzip .\master.zip
Set-ExecutionPolicy Bypass -Scope Process -Force; powershell ${HOME}\Downloads\fonts-master\install.ps1

Set Ubuntu Mono derivative Powerline as your font:

{
    "colorScheme": "Solarized Dark",
    "commandline": "powershell.exe",
    "fontFace": "Ubuntu Mono derivative Powerline",
    "guid": "{61c54bbd-c2c6-5271-96e7-009a87ff44bf}",
    "hidden": false,
    "name": "Windows PowerShell"
},

Source:
https://github.com/powerline/powerline

https://docs.microsoft.com/en-us/windows/terminal/tutorials/powerline-setup

#Pulumi with #GitHub Actions to provision a cluster on #AWS with #EKS

Anderson Gama — Fri, 08 Jan 2021 10:36:42 +0000

This is a model for creating in Python an EKS environment with the AWS provider on Pulumi using GitHub Actions.

Access "https://github.com/smashse/pulumi-iac-eks"
Click in "Use this template"
Create a new repository from template "pulumi-iac-eks"(example "pulumi-iac-eks") and chose as "Private"

Install AWS (Optional)

cd /tmp
ssh-keygen -f pulumi_eks_py_access
curl "<https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip>" -o "awscliv2.zip"
unzip awscliv2.zip && sudo ./aws/install
aws configure
aws ec2 import-key-pair --public-key-material "$(cat pulumi_eks_py_access.pub | base64)" --key-name pulumi_eks_py_access --region us-west-2 --profile yourprofile

Download the PULUMI template

mkdir -p $HOME/Pulumi
cd $HOME/Pulumi
git clone https://github.com/yourgithubuser/pulumi-iac-eks.git
cd pulumi-iac-eks

Install Pulumi on Linux by running the installation script:

curl -fsSL https://get.pulumi.com | sh && bash

Install Python VirtualEnv:

sudo apt -y install python3-virtualenv

Create a "pulumi_eks_py" project:

cd $HOME/Pulumi/pulumi-iac-eks/pulumi_eks_py

Note: If you want to change the name given to Kubernetes cluster, execute the command below in the template folder.

sed -i "s/"template-"/"desiredname-"/g" *.py

Install Python Requirements

python3 -m venv venv
source venv/bin/activate
python -m pip install --upgrade pip setuptools wheel
python -m pip install -r requirements.txt

Perform an initial deployment, run the following commands:

pulumi login
pulumi stack init pulumi_eks_py

Set AWS_PROFILE:

pulumi config set aws:profile yourprofile

Set AWS_REGION:

pulumi config set aws:region us-west-2

Review the "pulumi_eks_py" project

pulumi preview

Enable Workflow

cd $HOME/Pulumi/pulumi-iac-eks/.github/workflows

mv pull_request.yml.template pull_request.yml
mv push.yml.template push.yml

Environment Variables

There are a number of Environment Variables that can be set to interact with the action:

By default, Pulumi will try to connect to the Pulumi SaaS. For this to happen, the GitHub Action needs to be passed a "PULUMI_ACCESS_TOKEN".

Amazon Web Services (AWS)

For AWS, you'll need to create or use an existing IAM user for your action. Please see the Pulumi documentation page for pointers to the relevant AWS documentation for doing this.

As soon as you have an AWS user in hand, you'll set the environment variables "AWS_ACCESS_KEY_ID" and "AWS_SECRET_ACCESS_KEY" using GitHub Secrets, and then consume them in your action.

Note: Go to Settings> Secrets and add "PULUMI_ACCESS_TOKEN", "AWS_ACCESS_KEY_ID" and "AWS_SECRET_ACCESS_KEY" as new repository secret.

Commit the changes

cd $HOME/Pulumi/pulumi-iac-eks/

git add *
git add .github/workflows/*
git add .pulumi/*
git add pulumi_eks_py/*
git commit -m "pulumi-iac-eks"
git push

Access EKS Kubernetes cluster

sudo snap install kubectl --classic
aws eks list-clusters --region us-west-2 --profile yourprofile
aws eks --region us-west-2 --profile yourprofile update-kubeconfig --name $(pulumi stack output cluster-name)
kubectl get po --all-namespaces

Destroy the "pulumi_eks_py" project

cd $HOME/Pulumi/pulumi-iac-eks/pulumi_eks_py
pulumi destroy

Remove the "pulumi_eks_py" project from Stack

cd $HOME/Pulumi/pulumi-iac-eks/pulumi_eks_py
pulumi stack rm pulumi_eks_py

Source:

https://www.pulumi.com/docs/get-started/

https://www.pulumi.com/docs/reference/pkg/

https://www.pulumi.com/docs/intro/concepts/state/

https://www.pulumi.com/docs/guides/continuous-delivery/github-actions/

https://github.com/pulumi/actions

Install KUBERNETES (EKS-D)

Anderson Gama — Mon, 07 Dec 2020 17:38:40 +0000

Install KUBERNETES (EKS-D)

OS: UBUNTU

Install Kubectl

sudo snap install kubectl --classic

Install Amazon EKS Distro (EKS-D)

sudo snap install eks --classic --edge

eks (1.18/edge) v1.18.9 from Canonical✓ installed

Check the status

eks status --wait-ready

eks is running
high-availability: no
  datastore master nodes: 127.0.0.1:19001
  datastore standby nodes: none

Give execution permission to the current user

sudo usermod -a -G eks $USER
sudo chown -f -R $USER ~/.kube

Export the current EKS-D configuration information for use with Kubectl

mkdir -p $HOME/.kube
sudo eks config > .kube/config

Inspect the installation

eks inspect | grep running

  Service snap.eks.daemon-cluster-agent is running
  Service snap.eks.daemon-containerd is running
  Service snap.eks.daemon-apiserver is running
  Service snap.eks.daemon-apiserver-kicker is running
  Service snap.eks.daemon-control-plane-kicker is running
  Service snap.eks.daemon-proxy is running
  Service snap.eks.daemon-kubelet is running
  Service snap.eks.daemon-scheduler is running
  Service snap.eks.daemon-controller-manager is running

Check the version of Containerd

eks ctr --version

ctr github.com/containerd/containerd v1.3.7

Server / Client version

eks ctr version

Client:
  Version:  v1.3.7
  Revision: 8fba4e9a7d01810a393d5d25a3621dc101981175

Server:
  Version:  v1.3.7
  Revision: 8fba4e9a7d01810a393d5d25a3621dc101981175
  UUID: 339017b3-570e-43bd-a528-4a08123868ca

Access Kubernetes

EKS-D comes with its own version of Kubectl to access Kubernetes. Below we will cover 2 different forms of access to view the "nodes" and "services":

Conventional

eks kubectl get nodes

NAME           STATUS   ROLES    AGE   VERSION
myuser         Ready    <none>   33m   v1.18.9-eks-1-18-1

eks kubectl get services

NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.152.183.1   <none>        443/TCP   33m

Recommended

kubectl get nodes

NAME           STATUS   ROLES    AGE   VERSION
myuser         Ready    <none>   33m   v1.18.9-eks-1-18-1

kubectl get services

NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.152.183.1   <none>        443/TCP   33m

Note: As shown, the 2 forms have the same result, but from this point on we will only use the "Recommended" with Kubectl.

Deploy a test application

kubectl create deployment nginx --image=nginx

deployment.apps/nginx created

kubectl get pods

NAME                    READY   STATUS    RESTARTS   AGE
nginx-f89759699-2w75l   1/1     Running   0          33s

Starting and stopping EKS-D

EKS-D will continue to run until you decide to stop it. You can stop and start with the commands below:

sudo eks stop

Stopped.

sudo eks start

Started.

Basic information of your Kubernetes cluster

Server/Client version

kubectl version --short=true

Client Version: v1.19.4
Server Version: v1.18.9-1+c787d4d0c397b8

Cluster information

kubectl cluster-info

Kubernetes master is running at https://192.168.254.100:16443
CoreDNS is running at https://192.168.254.100:16443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://192.168.254.100:16443/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy

Configuration information

kubectl config view

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://192.168.254.100:16443
  name: microk8s-cluster
contexts:
- context:
    cluster: microk8s-cluster
    user: admin
  name: microk8s
current-context: microk8s
kind: Config
preferences: {}
users:
- name: admin
  user:
    token: REDACTED

Note: To view the access token, use the "--flatten=true" option.

View the nodes

kubectl get nodes -w

NAME           STATUS   ROLES    AGE   VERSION
myuser         Ready    <none>   33h   v1.18.9-eks-1-18-1

Information about a particular node

kubectl describe node myuser

Source:

https://microk8s.io/

https://microk8s.io/docs/commands

https://snapcraft.io/eks

https://snapcraft.io/kubectl

https://ubuntu.com/blog/install-amazon-eks-distro-anywhere

Another IaC tool ...

Anderson Gama — Wed, 07 Oct 2020 21:33:47 +0000

Modern Infrastructure as Code.

Create, deploy, and manage infrastructure on any cloud using familiar programming languages and tool

In this example we will use Pulumi Crosswalk for AWS to easily create a Topic, Queues and a User with access permissions to them. Pulumi Crosswalk for AWS is a collection of libraries that use automatic well-architected best practices to make common infrastructure-as-code tasks in AWS easier and more secure.

Install Pulumi on Linux by running the installation script:

curl -fsSL https://get.pulumi.com | sh

Install Node.js:

sudo snap install node --classic

Create a "pulumi_my" project:

mkdir pulumi_my && cd pulumi_my && pulumi new aws-typescript --emoji --generate-only

nano -c index.ts

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";

// Create an AWS resource (IAM)
const mypolicy = new aws.iam.Policy("my_policy", {
  name: "my_policy",
  policy: JSON.stringify({
    "Version": "2012-10-17",
    "Statement": [
      {
        "Action": "sns:*",
        "Effect": "Allow",
        "Resource": "arn:aws:sns:my-region:my-account:my_topic"
      },
      {
        "Action": "sqs:*",
        "Effect": "Allow",
        "Resource": "arn:aws:sqs:my-region:my-account:my_queue"
      },
      {
        "Action": "sqs:*",
        "Effect": "Allow",
        "Resource": "arn:aws:sqs:my-region:my-account:my_queue_dlq"
      }
    ]
  })
});

const myuser = new aws.iam.User("my_user", {
  name: "my_user",
});

const policyAttachment = new aws.iam.PolicyAttachment("my_policy_attachment", {
  name: "my_policy_attachment",
  users: [myuser],
  policyArn: mypolicy.arn
});

// Create an AWS resource (SNS)
const myTopic = new aws.sns.Topic("my_topic", {
  name: "my_topic",
  deliveryPolicy: `{
  "http": {
    "defaultHealthyRetryPolicy": {
      "numRetries": 3,
      "numNoDelayRetries": 0,
      "minDelayTarget": 20,
      "maxDelayTarget": 20,
      "numMinDelayRetries": 0,
      "numMaxDelayRetries": 0,
      "backoffFunction": "linear"
    },
    "disableSubscriptionOverrides": false
  }
}
`,
  kmsMasterKeyId: "alias/aws/sns",
  tags: {
    Environment: "myproduction",
  },
});

// Create an AWS resource (SQS)
const myQueueDlq = new aws.sqs.Queue("my_queue_dlq", {
  name: "my_queue_dlq",
  delaySeconds: 90,
  maxMessageSize: 2048,
  messageRetentionSeconds: 86400,
  receiveWaitTimeSeconds: 10,
  policy: JSON.stringify({
    "Version": "2012-10-17",
    "Id": "arn:aws:sqs:my-region:my-account:my_queue_dlq/SQSDefaultPolicy",
    "Statement": [
      {
        "Sid": "__owner_statement",
        "Action": "sqs:*",
        "Effect": "Allow",
        "Resource": "arn:aws:sqs:my-region:my-account:my_queue_dlq",
        "Principal": {
          "AWS": [
            "arn:aws:iam::my-account:user/my_user"
          ]
        }
      }
    ]
  }),
  tags: {
    Environment: "myproduction",
  },
}, { dependsOn: [myuser, mypolicy, policyAttachment] });

const myQueue = new aws.sqs.Queue("my_queue", {
  name: "my_queue",
  delaySeconds: 90,
  maxMessageSize: 2048,
  messageRetentionSeconds: 86400,
  receiveWaitTimeSeconds: 10,
  policy: JSON.stringify({
    "Version": "2012-10-17",
    "Id": "arn:aws:sqs:my-region:my-account:my_queue/SQSDefaultPolicy",
    "Statement": [
      {
        "Sid": "__owner_statement",
        "Action": "sqs:*",
        "Effect": "Allow",
        "Resource": "arn:aws:sqs:my-region:my-account:my_queue",
        "Principal": {
          "AWS": [
            "arn:aws:iam::my-account:user/my_user"
          ]
        }
      }
    ]
  }),
  //redrivePolicy: "{\"deadLetterTargetArn\":\"arn:aws:sqs:my-region:my-account:my_queue_dlq\",\"maxReceiveCount\":\"4\"}",
  tags: {
    Environment: "myproduction",
  },
}, { dependsOn: [myuser, mypolicy, policyAttachment, myQueueDlq] });


export const iamUserName = myuser.id;
export const iamAccesskeyID = myAccessKey.id;
export const iamAccesskeySecret = myAccessKey.sesSmtpPasswordV4;
export const arnPolicy = mypolicy.id;
export const arnTopic = myTopic.id;
export const urlQueueDlq = myQueueDlq.id;
export const urlQueue = myQueue.id;

Change "my-region" and "my-account"

Replace "my-region" with the region of your choice, for example, if you wanted to use "us-east-2" AKA "Ohio" it would look like below:

sed -i "s/"my-region"/"us-east-2"/g" index.ts

Replace "my-account" with your AWS account ID without the "-", for example if it were "5555-5555-5555" it would look like below:

sed -i "s/"my-account"/"555555555555"/g" index.ts

Perform an initial deployment, run the following commands:

npm install

pulumi stack init

Review the "pulumi_my" project

pulumi preview

Set AWS_PROFILE:

pulumi config set aws:profile my-profile

Set AWS_REGION:

pulumi config set aws:region us-east-2

Deploy the Stack

pulumi up

Destroy the "pulumi_my" project

pulumi destroy

Remove the "pulumi_my" project from Stack

pulumi stack rm dev

Source:

https://www.pulumi.com/docs/get-started/

https://www.pulumi.com/docs/guides/crosswalk/aws/

https://www.pulumi.com/docs/reference/pkg/

https://www.pulumi.com/docs/intro/concepts/state/

Terraform + GCP + MicroK8s

Anderson Gama — Wed, 12 Aug 2020 14:23:43 +0000

MicroK8s allows you to specify the amount of join and the token that will be used, then it opens up the opportunity for some interesting things, just imagine easily provisioning in Terraform an infrastructure with a simple multi-node cluster?

I showed in a previous post on #Linkedin the configuration "#MicroK8s+#K9S+#Octant": https://www.linkedin.com/posts/andersongama_microk8sk9soctant-sudo-apt-y-install-activity-6697607973889425408-9Gm-

Based on that thought up there you would define the "cloud-init"/"user_data" of "control-ship" that during provisioning will have the IP "10.158.0.2" that it should have the following token "868f88ee477f893de65c99d906e767dcaf59ce10fe30795ef9b2d11af4faefa" that was generated with a:

$ date | sha256sum | cut -f 1 -d -

Just out of curiosity if you were going to do it manually it would be:

$ sudo microk8s add-node --token-ttl 999 --token `date | sha256sum | cut -f 1 -d -`

For the "cargo-ship" nodes to be created only after the "control-ship" in the provisioning of instances, just put "depends_on = [aws_instance.control_instance]" if it is AWS and "depends_on = [google_compute_instance.control_instance]" if it is GCP.

For the control-ship:

#cloud-config
#control-ship
runcmd:
 - sudo echo "DEBIAN_FRONTEND=noninteractive" >> /etc/environment
 - sudo source /etc/environment && source /etc/environment
 - sudo apt update --fix-missing
 - sudo apt -y snap snapd
 - sudo snap refresh
 - sudo snap install microk8s --classic
 - sudo microk8s status --wait-ready
 - sudo microk8s add-node --token-ttl 999 --token 868f88ee477f893de65c99d906e767dcaf59ce10fe30795ef9b2d11af4faefa

For cargo-ship:

#cloud-config
#cargo-ship
runcmd:
 - sudo echo "DEBIAN_FRONTEND=noninteractive" >> /etc/environment
 - sudo source /etc/environment && source /etc/environment
 - sudo apt update --fix-missing
 - sudo apt -y snap snapd
 - sudo snap refresh
 - sudo snap install microk8s --classic
 - sudo microk8s status --wait-ready
 - sudo microk8s join 10.158.0.2:25000/868f88ee477f893de65c99d906e767dcaf59ce10fe30795ef9b2d11af4faefa

Note: If you found it interesting, you can take the template for #GCP "https://github.com/smashse/iac-public/tree/master/IAC/GCP/template_terraform_gcp_microk8s" and execute it inside the folder:

$ sed -i "s/template/desiredname/g" *.*

Instructions for setting up your environment with GCP on Linux: https://github.com/smashse/iac-public/tree/master/IAC/GCP

DEV Community: Anderson Gama

🚀 Setting Up a Kubernetes Observability Proof of Concept (POC) with MicroK8s and DataDog

Proof of Concept (POC) Requirements

1. Install Multipass

2. MicroK8s Setup

2.1 Create a MicroK8s Template

2.2 Create a MicroK8s Instance

2.3 Export MicroK8s Configuration for Kubectl

2.3.1 Create a Directory for Kubernetes Configurations

2.3.2 Export MicroK8s Configuration

2.3.3 Set the Exported Configuration as Default for Kubectl

2.3.4 Install Kubectl

2.3.5 Install Helm

2.4 Add an IP Alias for MicroK8s

2.5 Access MicroK8s via URL

3. DataDog Setup

US1 (US Region - Default)

US3 (US3 Region)

US5 (US5 Region)

Key Differences:

How to Choose the Right Region:

Note:

3.1 Getting Started with DataDog Free Tier

3.2 Install the Datadog Agent

3.2.1 Installing on Kubernetes

3.2.1.1 Install the Datadog Operator

3.2.1.2 Configure datadog-agent.yaml

3.2.1.3 Deploy the Datadog Agent

3.2.1.4 Confirm Agent Installation

3.2.2.1 Add the Datadog Helm Repository

3.2.2.2 Configure datadog-values.yaml

3.2.2.3 Deploy the Datadog Agent

3.2.2.4 Confirm Agent Installation

Multipass + Cloud-Init

Multipass

Install Multipass

Debian in Multipass

Rocky in Multipass

Matriz de Roles y Responsabilidades

Roles and Responsibilities Matrix

What is the role of an SRE in a squad?

What is the role of a Frontend Developer in a squad?

What is the role of a Backend Developer in a squad?

Matriz de Papéis e Responsabilidades

Qual é o papel de um SRE em uma squad?

Qual é o papel de um Desenvolvedor Frontend em uma squad?

Qual é o papel de um Desenvolvedor Backend em uma squad?

DevOps Environment on MacOS

Install Homebrew

Change default shell to bash

Install XCode

Install Python

Install Firefox ESR

Install Google Chrome

Install VSCodium

Install Visual Studio Code

Install Kubectl

Install Krew and Neat

Krew

Neat

Install Helm

Install Lens

Install Kubenav

Install Terraform

Install Terraform-Docs

Install AWScli

Install Azure-cli

Install Google Cloud SDK

Install Multipass

Install Podman

Install Rancher Desktop

Install ITerm2

Install Powerline-Go

Multipass, Microk8s, Prometheus and Grafana

Requirements for this POC

Install Multipass

MicroK8s

Create a MicroK8s template

Create a MicroK8s instance

Export the current MicroK8s configuration for use with Kubectl

3.2.1.2 Configure `datadog-agent.yaml`

3.2.2.2 Configure `datadog-values.yaml`