How to deploy Sentry on a Kubernetes cluster with an external custom ingress

Aboubacar Doucoure — Wed, 04 Sep 2024 15:07:36 +0000

Context

Sentry can be installed on a Kubernetes cluster with the help of the official Sentry Helm chart. Here's how.

Sentry is a developer-first error tracking and performance monitoring platform that helps developers see what actually matters, solve quicker, and learn continuously about their applications.

Pre-requisites

A working Kubernetes installation
A distributed block storage for Kubernetes (I am using Longhorn on Rancher)
Cert-manager or equivalent
Nginx or equivalent
A working helm tooling

Custom ingress

I could not make the included ingress configuration work. Here's a custom ingress that targets the -sentry-web service to the outside world.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: "{{ .Release.Name }}-sentry"
  annotations:
    nginx.ingress.kubernetes.io/use-regex: "true"  
spec:
  tls:
  - hosts:  
    - sentry.mywebsite.dev  
    secretName: "sentry.mywebsite.dev-tls"
  rules:
  - host: sentry.mywebsite.dev
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: "{{ .Release.Name }}-sentry-web"
            port: 
              number: 9000

Add "nginx.ingress.kubernetes.io/use-regex: "true"" as per the official Sentry documentation:

Note: if you are using NGINX Ingress, please set this annotation on your ingress : nginx.ingress.kubernetes.io/use-regex: "true". If you are using** additionalHostNames the nginx.ingress.kubernetes.io/upstream-vhostannotation might also come in handy. It sets the Host **header to the value you provide to avoid CSRF issues.

Install sentry

These are the helm chart values needed to kickstart the sentry installation:

  kafka:
    enabled: true
  user:
    create: true
    email: <REDACTED>
    password: <REDACTED>
  asHook: true
  sentry:
    singleOrganization: false
    worker:
      replicas: 2  
  auth:
    register: false
  hooks:
    activeDeadlineSeconds: 6500
  ingress:
    enabled: false  
  nginx:
    enabled: false
  clickhouse:
    enabled: true
    clickhouse:
      replicas: "1"  
  postgresql:
    enabled: true

Bonus: use sentry in a helm chart

Most of the examples in this tutorial were made for a custom tooling chart that includes sentry. Here's the sentry version and the repository used:

...
- name: sentry
    alias: sentry
    version: 23.x
    repository: https://sentry-kubernetes.github.io/charts
    condition: sentry.enabled
...

Voilà! You have a fully working Sentry installation.

Consumption

Here's a quick Loki view at how much Sentry consumes of my cluster's resource with no app being monitored.

I will not provide the exact details of every measurement as the cluster's name is redacted, but as you can see, Sentry taxes a heavy toll on the cluster's resources. At rest it takes up to 13Gb of RAM!

Inspirations and references

https://raslasarslas.medium.com/how-to-deploy-sentry-on-a-kubernetes-cluster-using-helm-600db31d4486

Use minio as an external object storage with Gitlab

Aboubacar Doucoure — Wed, 04 Sep 2024 04:32:20 +0000

Context

Gitlab can be installed on a Kubernetes cluster with the help of the official Gitlab Helm chart. Here I install it on a Rancher managed Kubernetes cluster with the usual griefs that come with the installation of such a behemoth: optimizing resources, picking the right subchart to install and how to install it. The installation is notoriously tedious given the substantial amount of subcharts and options, not counting the optimization needed to fit it in a resource-constrained cluster. Everything was working fine until we had a cluster issue with nodes crashing and I had to reinstall Gitlab from the custom chart I created. The installation only kept the gitaly, postgresql and redis PVCs, and to my devopsy sorrow, the minio storage was gone. There is no way around it, minio has to be installed seprately for a production ready self managed Gitlab. Here's how.

Pre-requisites

A working Kubernetes installation and a distributed block storage for Kubernetes (I am using Longhorn on Rancher)
A working helm tooling
Needles to say, a sufficiently provisioned cluster with preferably a backup and restoration routine

Install a minio chart

We'll use a Bitnami chart to install a standalone Minio instance:

minio-external:
  mode: standalone
  enabled: true
  persistence:
    enabled: true
    size: 10Gi
    annotations:
      helm.sh/resource-policy: keep
  auth:
    rootUser: <REDACTED>
    rootPassword: <REDACTED>
  provisioning:
    enabled: true
    users:
      - username: <REDACTED>
        password: <REDACTED>
        disabled: false
        policies:
          - readwrite
          - consoleAdmin
          - diagnostics  
        setPolicies: false
  accessKey:
    password: <REDACTED>
  secretKey:
    password: <REDACTED>
  # volumePermissions:
  #   enabled: true
  defaultBuckets: default,gitlab-registry-storage,gitlab-lfs,gitlab-artifacts,gitlab-uploads,gitlab-packages,gitlab-mr-diffs,gitlab-terraform-state,gitlab-ci-secure-files,gitlab-dependency-proxy,gitlab-pages

Use this configuration to kickstart a Minio instance with the buckets needed by Gitlab (defaultBuckets). These buckets are needed by different services of Gitlab and they will be mentionned in appConfig later.

The default minio values for this helm chart are available [here](# charts/bitnami/minio/values.yaml at main · bitnami/charts · GitHub).

Disable managed minio

Since we are using an external Minio instance we do not need the Gitlab managed one anymore. In your Gitlab values file disable minio:

global:
  ...
  minio:
    enabled: false

⚠️ This will delete your Gitlab Minio instance and all its data. Please proceed with caution. Consider backuping your data and migrating it afterwards using your tool of choice. I would use minio command line.

Create a secret

apiVersion: v1  
kind: Secret
metadata:  
    name: gitlab-object-storage
stringData:
  connection: |
    provider: AWS
    region: us-east-1
    aws_access_key_id: <REDACTED>
    aws_secret_access_key: <REDACTED>
    endpoint: "gitlab-minio-external:9000"

Connect Gitlab to the new Minio instance

  global:
    registry:
      bucket: gitlab-registry-storage
    appConfig: 
      ...  
      object_store:
        enabled: true
        proxy_download: true
        storage_options:
          {}
          # server_side_encryption:
          # server_side_encryption_kms_key_id
        connection:
          secret: gitlab-object-storage
          key: connection
      lfs:
        enabled: true
        proxy_download: true
        bucket: gitlab-lfs   
      artifacts:
        enabled: true
        proxy_download: true
        bucket: gitlab-artifacts
      uploads:
        enabled: true
        proxy_download: true
        bucket: gitlab-uploads
      packages:
        enabled: true
        proxy_download: true
        bucket: gitlab-packages
      externalDiffs:
        enabled: true
        when:
        proxy_download: true
        bucket: gitlab-mr-diffs
      terraformState:
        enabled: true
        bucket: gitlab-terraform-state
      ciSecureFiles:
        enabled: true   
        bucket: gitlab-ci-secure-files
        # connection:
        #   secret: gitlab-object-storage
      dependencyProxy:
        enabled: true
        proxy_download: true
        bucket: gitlab-dependency-proxy
      pages:
        enabled: true
        proxy_download: true
        bucket: gitlab-pages

Launch your gitlab installation and it should work.

Voilà!

DEV Community: Aboubacar Doucoure

How to deploy Sentry on a Kubernetes cluster with an external custom ingress

Context

Pre-requisites

Custom ingress

Install sentry

Bonus: use sentry in a helm chart

Consumption

Inspirations and references

Use minio as an external object storage with Gitlab

Context

Pre-requisites

Install a minio chart

Disable managed minio

Create a secret

Connect Gitlab to the new Minio instance

Inspirations and references