DEV Community: Rodrigo Burgos

Running Anchor Tests on GitHub Actions Without Losing Your Mind

Rodrigo Burgos — Tue, 10 Mar 2026 06:00:41 +0000

Setting up anchor test on GitHub Actions sounds like a weekend task. It took me a full week of debugging to get it right. Here's
everything I found so far, so you don't have to.

The context

I'm building an open source cross-chain bridge between Ethereum and Solana. The Solana side uses Anchor 0.31.1. At some point I
needed CI — integration tests running automatically on every push, no manual anchor test on my machine.

Simple enough, right?

Problem 1: Agave 3.x and io_uring

The first approach: install the Solana CLI inside the CI runner directly.

  - run: sh -c "$(curl -sSfL https://release.anza.xyz/stable/install)"

stable at the time of writing resolves to Agave 3.1.x. The install completes fine. Then anchor test starts the local validator and
immediately panics:

  thread 'main' panicked at fs/src/dirs.rs:27:9:
  assertion failed: io_uring_supported()

Agave 3.x added a hard dependency on io_uring, a Linux kernel feature for async I/O. GitHub Actions runners run on kernels where
io_uring is either disabled or not available.

Fix: use Agave 2.x. v2.1.21 is the latest in the 2.1 series and works without io_uring.


  ENV SOLANA_VERSION=v2.1.21
  RUN sh -c "$(curl -sSfL https://release.anza.xyz/${SOLANA_VERSION}/install)"

Problem 2: GLIBC mismatch

Next issue: the anchor binary itself.

avm install 0.31.1 downloads a pre-built binary from GitHub releases. That binary was compiled on a system with GLIBC 2.38/2.39.
Ubuntu 22.04 ships with GLIBC 2.35.

   /root/.avm/bin/anchor-0.31.1: /lib/x86_64-linux-gnu/libm.so.6:
     version `GLIBC_2.38' not found

Two fixes needed:

Switch the base image to Ubuntu 24.04 (ships with GLIBC 2.39)
Compile anchor from source instead of using avm. A binary compiled inside the container will always be compatible with the container's GLIBC.

FROM ubuntu:24.04

  RUN cargo install \
    --git https://github.com/coral-xyz/anchor \
    --tag v0.31.1 anchor-cli --locked

This makes the build slower (adds ~5 min to the image build), but the image is built once and cached. Runtime CI stays fast.

Problem 3: Container environment quirks

When GitHub Actions runs a job inside a container, it overrides some environment variables. Two things break silently:

cargo can't find its toolchain:

error: rustup could not choose a version of cargo to run,
because one wasn't specified explicitly, and no default is configured.

Fix: explicitly set CARGO_HOME and RUSTUP_HOME in the job env:

  env:
    CARGO_HOME: /root/.cargo
    RUSTUP_HOME: /root/.rustup

solana-keygen writes to the wrong place:

The keypair must exist at ~/.config/solana/id.json. Inside a GitHub Actions container, HOME is /github/home, not /root. If you
hardcode /root/.config/solana/id.json it won't be found by Anchor.

  - name: Generate keypair
    run: |
      mkdir -p $HOME/.config/solana
      solana-keygen new \
        --outfile $HOME/.config/solana/id.json \
        --no-bip39-passphrase --force

Problem 4: Test validator startup time

The local validator takes longer to start in a CI runner than on a dev machine. Without configuring a startup wait, anchor test
will fail with:

Unable to get latest blockhash. Test validator does not look started.

Add this to your Anchor.toml:

  [test]
  startup_wait = 60000

Problem 5: Event listeners hang forever

This was the hardest one to diagnose. Tests that use program.addEventListener work fine locally but hang indefinitely in CI:

  const eventPromise = new Promise<any>((resolve) => {
    const listenerId = program.addEventListener("TokenSent", (event) => {
      program.removeEventListener(listenerId);
      resolve(event);
    });
  });

  await program.methods.bridgeSend(...).rpc();
  const event = await eventPromise; // hangs forever in CI

The reason: addEventListener opens a WebSocket subscription to the local validator. In containerized environments, this
subscription is established but log notifications are never delivered. The promise never resolves. Mocha's timeout is 1000s by
default, so the test runner just sits there.

Fix: fetch the transaction logs directly after rpc() and parse the events from them.

  import { BorshCoder } from "@coral-xyz/anchor";
  import IDL from "../target/idl/bridge.json";

  const eventCoder = new BorshCoder(IDL as any).events;

  async function getConfirmedTx(sig: string) {
    for (let i = 0; i < 10; i++) {
      const tx = await provider.connection.getTransaction(sig, {
        commitment: "confirmed",
        maxSupportedTransactionVersion: 0,
      });
      if (tx) return tx;
      await new Promise((r) => setTimeout(r, 1000));
    }
    throw new Error(`Transaction ${sig} not found after retries`);
  }

  function parseEvents(logMessages: string[]) {
    return logMessages
      .filter((log) => log.startsWith("Program data: "))
      .map((log) => {
        try {
          return eventCoder.decode(log.slice("Program data: ".length));
        } catch {
          return null;
        }
      })
      .filter(Boolean);
  }

Then in the test:

  const sig = await program.methods.bridgeSend(...).rpc();
  const tx = await getConfirmedTx(sig);
  const events = parseEvents(tx.meta.logMessages);
  const event = events.find((e) => e.name === "TokenSent")?.data;

  assert.ok(event, "TokenSent event not found");
  assert.equal(event.amount.toNumber(), 100_000);

Two notes on this approach:

getTransaction can return null briefly after rpc() returns, even after confirmation — hence the retry loop.
Use new BorshCoder(IDL) directly rather than program.coder. In some container configurations program.coder.events.decode returns null even when the discriminator matches. Loading the IDL directly is reliable.

Also add "resolveJsonModule": true to your tsconfig.json to enable the JSON import.

The pre-built image

To avoid reinstalling all of this on every CI run, I packaged it into a Docker image:

docker pull burgossrodrigo/anchor-build:0.31.1

Full integration.yml example:

solana:
    runs-on: ubuntu-latest
    container:
      image: burgossrodrigo/anchor-build:0.31.1

    env:
      CARGO_HOME: /root/.cargo
      RUSTUP_HOME: /root/.rustup

    steps:
      - uses: actions/checkout@v4

      - name: Cache build artifacts
        uses: actions/cache@v4
        with:
          path: contracts/solana/target
          key: ${{ runner.os }}-anchor-${{ hashFiles('contracts/solana/Cargo.lock') }}

      - name: Generate keypair
        run: |
          mkdir -p $HOME/.config/solana
          solana-keygen new \
            --outfile $HOME/.config/solana/id.json \
            --no-bip39-passphrase --force

      - name: Fix blake3 compatibility
        working-directory: contracts/solana
        run: cargo update -p blake3 --precise 1.8.2

      - name: Fix indexmap compatibility
        working-directory: contracts/solana
        run: cargo update -p indexmap --precise 2.11.4

      - name: Run tests
        working-directory: contracts/solana
        run: anchor test

The blake3 and indexmap pins are required because cargo build-sbf uses a bundled Rust (1.79) that is incompatible with their
latest versions — blake3 1.8.3+ introduced edition2024, and indexmap 2.12+ has an MSRV higher than 1.79.

Source

The full project — bridge contracts, backend relayer, CI setup — is open source:

github.com/burgossrodrigo/token_bridge

How to Make Your Rust Tests Run Faster in CI (A Practical Guide)

Rodrigo Burgos — Thu, 05 Mar 2026 00:22:03 +0000

Slow CI pipelines are often blamed on:

Heavy test suites
Complex integrations
Rust compilation time

But in many cases, the real issue is much simpler:

Your tests are not fully using the CPU available in the CI runner.
But in many cases, the real issue is much simpler:
Your tests are not fully using the CPU available in the CI runner.

Step 1 — Understand How cargo test Uses Threads

Rust’s test harness runs tests in parallel by default. However, in CI environments:

CPU limits may restrict available cores
Containers may expose fewer threads
The harness may default to 1 thread in constrained setups
You should never assume your CI is using all available CPUs.

Instead, verify it.

Step 2 — Check How Many CPUs Your Runner Has

Inside your CI job, run:

nproc

Example output:

This means the environment has 2 logical CPUs available. If you don’t explicitly configure thread usage, your tests might not use both.

Step 3 — Explicitly Set --test-threads

script:
  - cargo test -p my_crate module_a
  - cargo test -p my_crate module_b
  - cargo test -p my_crate module_c
  - cargo test -p my_crate module_d

Each invocation runs sequentially. To ensure each test run uses all available CPU cores, capture the number of CPUs dynamically:

script:
  - THREADS=$(nproc)
  - echo "Running tests with ${THREADS} threads"
  - cargo test -p my_crate module_a -- --test-threads=${THREADS}
  - cargo test -p my_crate module_b -- --test-threads=${THREADS}
  - cargo test -p my_crate module_c -- --test-threads=${THREADS}
  - cargo test -p my_crate module_d -- --test-threads=${THREADS}

Why the Double Dash (--) Is Important. The -- separator is critical. Everything before -- is interpreted by cargo. Everything after -- is passed to the test binary (Rust’s test harness).

--test-threads is a test harness argument — not a cargo argument. If you forget the separator, the flag won’t work.

Step 4 — Why Use $(nproc) Instead of a Fixed Number?

You could hardcode:

--test-threads=2

But that creates a hidden maintenance issue. If the runner changes from 2 CPUs to 4, your CI won’t scale automatically.

Using: THREADS=$(nproc) ensures:

Automatic adaptation
No future edits required
Better portability between environments

Step 5 — Make Sure Your Tests Are Safe to Parallelize

Parallel test execution requires test isolation. Your tests should:

Avoid global mutable state
Avoid shared in-memory singletons
Avoid reusing the same database instance
Avoid mutating global environment variables

A safe pattern is to instantiate dependencies per test:

fn create_test_repository() -> InMemoryRepository {
    InMemoryRepository::new()
}

[tokio::test]
fn example_test() {
    let repo = create_test_repository();
    // test logic here
}

Each test gets its own isolated state. When You Should Disable Parallelism. If a test suite depends on shared external state (for example, a real database instance), you may need to force sequential execution:

cargo test -- --test-threads=1

Apply this only to specific test groups that require it. Do not disable parallelism globally unless necessary.

Optional Optimization: Avoid Repeating Expensive Setup

Sometimes slow tests are caused by repeated expensive operations (for example, hashing, cryptographic setup, or large fixture generation). You can cache computed values safely using OnceLock:

use std::sync::OnceLock;

static PRECOMPUTED_VALUE: OnceLock<String> = OnceLock::new();

fn get_precomputed_value() -> String {
    PRECOMPUTED_VALUE
        .get_or_init(|| {
            expensive_operation()
        })
        .clone()
}

fn expensive_operation() -> String {
    // Simulate heavy work
    "computed_result".to_string()
}

This ensures:

The expensive operation runs only once
Tests remain deterministic
No unsafe global mutation occurs

However, always evaluate tradeoffs:

Does it significantly reduce runtime?
Does it add unnecessary complexity?
Is parallelism alone sufficient?

Often, proper thread configuration already solves most CI performance issues.

Expected Impact

If your CI was running tests effectively single-threaded on a multi-core runner, explicitly configuring --test-threads can:

Reduce test stage time dramatically
Improve resource utilization
Avoid unnecessary infrastructure upgrades

In many cases, improvements of 2–3x are realistic.

Final Checklist

If your Rust CI feels slow, verify the following:

How many CPUs does the runner expose? (nproc)
Are tests running in parallel?
Is --test-threads explicitly configured?
Are tests properly isolated?
Are expensive operations unnecessarily repeated?

Before rewriting your test suite or scaling infrastructure, make sure you are actually using the hardware available to you.

Api rust com pulumi IaC, k8s, gke, dns e CI no gitlab.

Rodrigo Burgos — Fri, 26 Sep 2025 15:32:16 +0000

O projetinho de hoje é construir uma api rust, com deploy no gke usando pulumi como iac. Nossa primeira abordagem será criar a nossa IaC usando pulumi.

IaC

Primeiro recurso é o ApiDeployment, não é o serviço propriamente dito mas possui algumas especificações do pod como recurso de máquina, portas (do serviço, observabilidade) e variáveis de ambiente:

func ApiDeployment(ctx *pulumi.Context, provider *kubernetes.Provider, apiLabels pulumi.StringMap, secret *corev1.Secret, image pulumi.StringInput) error {
    _, err := appsv1.NewDeployment(ctx, "api", &appsv1.DeploymentArgs{
        Metadata: &metav1.ObjectMetaArgs{
            Name: pulumi.String("api"),
        },
        Spec: &appsv1.DeploymentSpecArgs{
            Replicas: pulumi.Int(1),
            Selector: &metav1.LabelSelectorArgs{
                MatchLabels: apiLabels,
            },
            Template: &corev1.PodTemplateSpecArgs{
                Metadata: &metav1.ObjectMetaArgs{
                    Labels: apiLabels,
                },
                Spec: &corev1.PodSpecArgs{
                    Containers: corev1.ContainerArray{
                        &corev1.ContainerArgs{
                            Name:  pulumi.String("api"),
                            Image: image,
                            Ports: corev1.ContainerPortArray{
                                &corev1.ContainerPortArgs{ContainerPort: pulumi.Int(8080)},
                                &corev1.ContainerPortArgs{ContainerPort: pulumi.Int(3001), Name: pulumi.String("metrics")},
                            },
                            Resources: &corev1.ResourceRequirementsArgs{
                                Requests: pulumi.StringMap{"cpu": pulumi.String("100m")},
                                Limits:   pulumi.StringMap{"cpu": pulumi.String("500m")},
                            },
                            Env: corev1.EnvVarArray{
                                &corev1.EnvVarArgs{
                                    Name: pulumi.String("TEST"),
                                    ValueFrom: &corev1.EnvVarSourceArgs{
                                        SecretKeyRef: &corev1.SecretKeySelectorArgs{
                                            Name: secret.Metadata.Name(),
                                            Key:  pulumi.String("TEST"),
                                        },
                                    },
                                },
                            },
                        },
                    },
                },
            },
        },
    }, pulumi.Provider(provider))

Proximo recurso é o serviço propriamente dito, mais focado no load balancer (atenção pra congruência das portas expostas).

func ApiLb(ctx *pulumi.Context, provider *kubernetes.Provider, apiLabels pulumi.StringMap) (*corev1.Service, error) {
    apiService, err := corev1.NewService(ctx, "api-lb", &corev1.ServiceArgs{
        Metadata: &metav1.ObjectMetaArgs{
            Name:   pulumi.String("api-lb"),
            Labels: apiLabels,
            Annotations: pulumi.StringMap{
                "cloud.google.com/neg":            pulumi.String(`{"ingress": true}`),
                "cloud.google.com/backend-config": pulumi.String(`{"default":"api-backendconfig"}`),
                "pulumi.com/skipAwait":            pulumi.String("true"),
            },
        },
        Spec: &corev1.ServiceSpecArgs{
            Selector: apiLabels,
            Ports: corev1.ServicePortArray{
                &corev1.ServicePortArgs{
                    Port:       pulumi.Int(80),
                    TargetPort: pulumi.Int(8080),
                },
            },
        },
    }, pulumi.Provider(provider))
    if err != nil {
        return nil, err
    }

    return apiService, nil
}

Este recurso diz respeito ao ingress, a classe do ingress, o gerenciador de certificado, nome do ip estático público e o o domínio são gerenciados aqui:

func ApiIngress(
    ctx *pulumi.Context,
    provider *kubernetes.Provider,
    apiService *corev1.Service,
    cert *apiextensions.CustomResource,
    ip *compute.GlobalAddress, // <— MUDEI: recebe o recurso inteiro
) (*networkingv1.Ingress, error) {

    ingress, err := networkingv1.NewIngress(ctx, "ingress", &networkingv1.IngressArgs{
        Metadata: &metav1.ObjectMetaArgs{
            Name: pulumi.String("ingress"),
            Annotations: pulumi.StringMap{
                "networking.gke.io/managed-certificates": pulumi.String("managed-cert"),
                "kubernetes.io/ingress.class":            pulumi.String("gce"),
                "pulumi.com/skipAwait":                   pulumi.String("true"),
                // <- Aqui vai o NOME do GlobalAddress:
                "kubernetes.io/ingress.global-static-ip-name": ip.Name,
            },
        },
        Spec: &networkingv1.IngressSpecArgs{
            IngressClassName: pulumi.StringPtr("gce"),
            Rules: networkingv1.IngressRuleArray{
                &networkingv1.IngressRuleArgs{
                    Host: pulumi.String("subdomio.dominio.com"),
                    Http: &networkingv1.HTTPIngressRuleValueArgs{
                        Paths: networkingv1.HTTPIngressPathArray{
                            &networkingv1.HTTPIngressPathArgs{
                                Path:     pulumi.String("/"),
                                PathType: pulumi.String("Prefix"),
                                Backend: &networkingv1.IngressBackendArgs{
                                    Service: &networkingv1.IngressServiceBackendArgs{
                                        Name: pulumi.String("api-lb"),
                                        Port: &networkingv1.ServiceBackendPortArgs{
                                            Number: pulumi.Int(80),
                                        },
                                    },
                                },
                            },
                        },
                    },
                },
            },
        },
    }, pulumi.Provider(provider), pulumi.DependsOn([]pulumi.Resource{apiService, cert, ip}))

    if err != nil {
        return nil, err
    }
    return ingress, nil
}

Já este recurso é uma outra camada de gerenciamento do certificado, especificações do gcp.

func ManagedCert(ctx *pulumi.Context, provider *kubernetes.Provider) (*apiextensions.CustomResource, error) {
    cert, err := apiextensions.NewCustomResource(ctx, "managed-cert", &apiextensions.CustomResourceArgs{
        ApiVersion: pulumi.String("networking.gke.io/v1"),
        Kind:       pulumi.String("ManagedCertificate"),
        Metadata: metav1.ObjectMetaArgs{
            Name:      pulumi.String("managed-cert"),
            Namespace: pulumi.String("default"),
        },
        OtherFields: kubernetes.UntypedArgs{
            "spec": pulumi.Map{
                "domains": pulumi.StringArray{
                    pulumi.String("subdominio.dominio.com"),
                },
            },
        },
    }, pulumi.Provider(provider))

    if err != nil {
        return nil, err
    }

    return cert, nil
}

Terceira camada de gerenciamento de recurso, especificação do gcp:

func NewRecordSet(ctx *pulumi.Context, managedZone *dns.ManagedZone, addr pulumi.StringInput) error {
    _, err := dns.NewRecordSet(ctx, "nomeclatura-arbitraria", &dns.RecordSetArgs{
        ManagedZone: managedZone.Name,
        Name:        pulumi.String("subdominio.dominio.com."),
        Type:        pulumi.String("A"),
        Ttl:         pulumi.Int(300),
        Rrdatas:     pulumi.StringArray{addr},
    })
    return err
}

Quarta camada de gerenciamento de dns

func ManagedZone(ctx *pulumi.Context) (*dns.ManagedZone, error) {
    managedZone, err := dns.NewManagedZone(ctx, "nomeclatura-arbitraria-para-zona", &dns.ManagedZoneArgs{
        DnsName:     pulumi.String("subdominio.dominio.com."),
        Description: pulumi.String("Managed zone for GKE ingress"),
    })
    if err != nil {
        return nil, err
    }
    return managedZone, nil
}

Declaração de IP público

func IngressIP(ctx *pulumi.Context) (*compute.GlobalAddress, error) {
    ip, err := compute.NewGlobalAddress(ctx, "nomeclatura-arbitraria-referenciando-ip", &compute.GlobalAddressArgs{
        AddressType: pulumi.String("EXTERNAL"),
        IpVersion:   pulumi.String("IPV4"),
        Name:        pulumi.String("nomeclatura-arbitraria-referenciando-ip"),
    })
    if err != nil {
        return nil, err
    }
    return ip, nil
}

Nossa main function com todas as declarações:

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
        cfg := config.New(ctx, "gcp")
        region := cfg.Require("region")
        image := config.Require(ctx, "apiImage")

        cluster, err := infra.Cluster(ctx, region)
        if err != nil {
            return err
        }

        node, err := infra.Node(region, ctx, cluster)
        if err != nil {
            return err
        }

        kubeconfig := utils.Kubeconfig(cluster)

        provider, err := infra.Provider(ctx, node, &kubeconfig)
        if err != nil {
            return err
        }

        apiLabels := service.ApiLabels()

        secret, err := utils.CreateAppSecret(ctx, provider)

        err = service.ApiDeployment(ctx, provider, apiLabels, secret, pulumi.String(image))
        if err != nil {
            return err
        }

        apiLb, err := service.ApiLb(ctx, provider, apiLabels)
        if err != nil {
            return err
        }

        // Cria o ManagedCertificate e o armazena para ser usado como dependência.
        managedCert, err := service.ManagedCert(ctx, provider)
        if err != nil {
            return err
        }

        // O Ingress agora recebe o ManagedCert como argumento,
        // garantindo a ordem de criação correta.
        // _, err = service.ApiIngress(ctx, provider, apiLb, managedCert)
        // if err != nil {
        //  return err
        // }
        //

        IP, err := utils.IngressIP(ctx)
        if err != nil {
            return err
        }

        _, err = service.ApiIngress(ctx, provider, apiLb, managedCert, IP)
        if err != nil {
            return err
        }

        err = utils.BackendConfig(ctx, provider)
        if err != nil {
            return err
        }

        // ingressIP := service.IngressIP(apiIngress)

        // ingressIP.ApplyT(func(ip string) error {
        //  ctx.Log.Info("Ingress IP: "+ip, nil)
        //  return nil
        // })

        managedZone, err := service.ManagedZone(ctx)
        if err != nil {
            return err
        }

        err = service.NewRecordSet(ctx, managedZone, IP.Address)
        if err != nil {
            return err
        }

        return nil

    })

}

Dockerfile

Não vou me aprofundar muito na api, aqui está o dockerfile performático com gerenciamento de cache:

# ===== Base (Rust + deps de build) =====
ARG RUST_VERSION=1.83-bookworm
FROM rust:${RUST_VERSION} AS rust-base
RUN apt-get update && apt-get install -y --no-install-recommends \
    pkg-config libssl-dev ca-certificates && \
    rm -rf /var/lib/apt/lists/*
WORKDIR /app

# ===== Planner (gera a recipe do cargo-chef) =====
FROM rust-base AS planner
RUN cargo install cargo-chef --locked
COPY Cargo.* ./
COPY src ./src
RUN cargo chef prepare --recipe-path recipe.json

# ===== Cache de deps (cozinha as dependências) =====
FROM rust-base AS cacher
RUN cargo install cargo-chef --locked
COPY --from=planner /app/recipe.json /app/recipe.json
RUN cargo chef cook --release --recipe-path /app/recipe.json

# ===== Build final do binário =====
FROM rust-base AS builder
# Opcional: garante lockfile se não estiver versionado
COPY . .
RUN [ -f Cargo.lock ] || cargo generate-lockfile
# Reaproveita cache de deps via layers do chef
RUN cargo build --release

# ===== Runtime (slim, com OpenSSL 3) =====
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
    libssl3 ca-certificates && \
    apt-get clean && rm -rf /var/lib/apt/lists/*
WORKDIR /app

COPY --from=builder /app/target/release/test_api /app/test_api

ENV PORT=8080
EXPOSE 8080
CMD ["/app/test_api"]

Gitlab CI/CD

Mandando a imagem pro artifact registry do gcp

push_test_api:
    stage: push_test_api
    tags: [temp]
    environment: { name: sandbox }
    rules:
        - if: '$CI_COMMIT_BRANCH == "sandbox" && $CI_PIPELINE_SOURCE == "push"'
          changes: ["test_api/**/*"]
          when: always
        - when: never
    before_script:
        - *before_gcp
    script: |
        set -euo pipefail
        IMAGE_TAG="${CI_COMMIT_SHORT_SHA}"
        REPO="us-central1-docker.pkg.dev/projeto/registry"
        IMAGE_NAME="test-api"
        IMAGE_URI="${REPO}/${IMAGE_NAME}:${IMAGE_TAG}"

        docker build -t "${IMAGE_URI}" -f test_api/Dockerfile test_api
        docker push "${IMAGE_URI}"

        # Pin por digest (opcional, recomendado)
        DIGEST=$(gcloud artifacts docker images describe "${IMAGE_URI}" --format="value(image_summary.digest)")
        echo "IMAGE_URI=${IMAGE_URI}"   >  image_env.txt
        echo "IMAGE_DIGEST=${DIGEST}"   >> image_env.txt
    artifacts:
        reports:
            dotenv: image_env.txt
    after_script:
        - rm -f gcloud-key.json || true

Atualizando o recurso através do pulumi

deploy_api_pulumi:
    stage: deploy_api_pulumi
    needs: ["push_test_api"]
    tags: [temp]
    environment: { name: sandbox }
    rules:
        - if: '$CI_COMMIT_BRANCH == "branch_name" && $CI_PIPELINE_SOURCE == "push"'
          changes: ["test_api/**/*"]
          when: always
        - when: never
    before_script:
        - *before_gcp
        # Pulumi + Go
        - curl -fsSL https://get.pulumi.com | sh
        - export PATH="$HOME/.pulumi/bin:$PATH"
        - mkdir -p "$CI_PROJECT_DIR/.tmp/go"
        - curl -sSL "https://go.dev/dl/go1.23.0.linux-amd64.tar.gz" | tar -xz -C "$CI_PROJECT_DIR/.tmp/go" --strip-components=1
        - export PATH="$CI_PROJECT_DIR/.tmp/go/bin:$PATH"
        # GKE auth plugin + kubectl
        - gcloud components install gke-gcloud-auth-plugin kubectl --quiet || true
        - export USE_GKE_GCLOUD_AUTH_PLUGIN=True
        - kubectl version --client=true
        # kubeconfig
        - gcloud container clusters get-credentials seu-cluster --region us-east1 --project seu-projeto
    script: |
        set -euo pipefail
        cd iac_secret_manager

        # usa o backend do Pulumi.yaml (gs://seu-bucket-gerenciamento-state)
        pulumi login

        export PULUMI_CONFIG_PASSPHRASE=""

        pulumi stack select dev

        # aponta a imagem imutável produzida no job anterior
        pulumi config set apiImage "${IMAGE_URI}"

        # ---- (opcional) injete configs/segredos que sua app usa ----
        # Exemplos: (defina as envs no GitLab CI e marque-as como masked/protected)
        # pulumi config set --secret TEST "$TEST"
"$AUTH_TOKEN_KEY_STAGING"
        # ------------------------------------------------------------

        pulumi up --yes
        kubectl rollout status deploy/api -n default --timeout=120s
    after_script:
        - rm -f gcloud-key.json || true

Garanta um SA (service account) com permissões necessárias para essa operação. Crie o bucket no gs para gerenciar o estado, se o gitlab for self hosted, prepare uma VM para computar essas operações (com bastante espaço pra armazenar o cache ou com algum mecanismo de limpeza periódico) e você terá uma CI/CD segura e performática para seu projeto :)

Deploy de uma api em rust em cloudrun através de pipeline com gitlab

Rodrigo Burgos — Mon, 08 Sep 2025 21:21:10 +0000

Post genuíno, em português e escrito por mim (gpt revisou hehehe). Tentarei ser o máximo objetivo possível mas sinta-se convidado a discutir abordagens ou tirar dúvidas por dm ou nos comentários.

A abordagem que utilizei para esta api é a de DDD (domain drive design). Não vou me demorar muito no desenvolvimento da api em sí, vou focar no m̀ain.rs e na árvore de arquivos.

use test_api::infra::handler::health::get_health_handler;
use tokio;
use warp::Filter;

#[tokio::main]
async fn main() {
    // Define the route
    let get_health_route = warp::path("health")
        .and(warp::get())
        .and_then(get_health_handler);

    // Start the warp server
    warp::serve(get_health_route)
        .run(([0, 0, 0, 0], 8080))
        .await;
}

├── Cargo.toml
├── Dockerfile
├── REAME.md
├── src
│   ├── domain
│   │   ├── entities
│   │   └── use_case
│   ├── infra
│   │   └── handler
│   ├── lib.rs
│   ├── main.rs
│   └── use_case
│       └── get_health
└── target
    ├── CACHEDIR.TAG
    └── debug
        ├── build
        ├── deps
        ├── examples
        └── incremental

15 directories, 6 files

Pontos importantes para focar são os paramêtros do run do warp/serve:

.run(([0, 0, 0, 0], 8080))

Para que o container seja acessado externamente.

Dockerfile

# Etapa 1: Builder com Rust + OpenSSL
FROM rust:latest AS builder

RUN apt-get update && apt-get install -y pkg-config libssl-dev

WORKDIR /usr/src/test_api

# Copia arquivos do crate
COPY Cargo.toml ./
COPY src ./src

# Gera lockfile e baixa dependências
RUN cargo generate-lockfile
RUN cargo build --release

# Etapa 2: Imagem final com compatibilidade glibc
FROM debian:bookworm-slim

# Instala OpenSSL e dependências mínimas de runtime
RUN apt-get update && apt-get install -y libssl3 ca-certificates && \
    apt-get clean && rm -rf /var/lib/apt/lists/*

WORKDIR /app

COPY --from=builder /usr/src/test_api/target/release/test_api .

ENV PORT=8080
EXPOSE 8080

CMD ["./test_api"]

Gitlab

stages:
    - deploy_cloud_run

deploy_cloud_run:
    stage: deploy_cloud_run
    tags: [temp]
    environment:
        name: dev
    rules:
        - if: '$CI_COMMIT_BRANCH == "dev" && $CI_PIPELINE_SOURCE == "push"'
          changes:
              - **/*
          when: always
        - when: never
    before_script:
        - mkdir -p "$CI_PROJECT_DIR/.tmp/google-cloud-sdk"
        - curl -sSL "https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-438.0.0-linux-x86_64.tar.gz" | tar -xz -C "$CI_PROJECT_DIR/.tmp/google-cloud-sdk" --strip-components=1
        - export PATH="$CI_PROJECT_DIR/.tmp/google-cloud-sdk/bin:$PATH"
        - echo "$SERVICE_ACCOUNT_JSON" | base64 -d > gcloud-key.json
        - gcloud auth activate-service-account --key-file=gcloud-key.json
        - export GCP_PROJECT_ID="seu-projeto"
        - gcloud config set project "$GCP_PROJECT_ID"
        - gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
    script:
        - |
            IMAGE_TAG="$(git rev-parse --short HEAD)"
            IMAGE_URI="us-central1-docker.pkg.dev/seu-repo/api/test-api:${IMAGE_TAG}"

            docker build \
              -t "$IMAGE_URI" \
              -f test_api/Dockerfile \
              test_api

            docker push "$IMAGE_URI"

            gcloud run deploy api-dev \
              --image "$IMAGE_URI" \
              --platform managed \
              --region us-central1 \
              --allow-unauthenticated \
              --project "$GCP_PROJECT_ID" \
              --set-env-vars "ENVIRONMENT=staging"
    after_script:
        - rm -f gcloud-key.json

Permissões qua service account necessita para rodar esse job:

gcloud projects add-iam-policy-binding seu-projeto \ --member="serviceAccount:service-account@project.iam.gserviceaccount.com" \ --role="roles/run.admin"

gcloud projects add-iam-policy-binding seu-projeto \ --member="serviceAccount:service-account@project.iam.gserviceaccount.com" \ --role="roles/viewer"

gcloud projects add-iam-policy-binding seu-projeto \ --member="serviceAccount:service-account@project.iam.gserviceaccount.com" \ --role="roles/iam.serviceAccountUser"

O json da SA deve ser armazenado como segredo do Gitlab.

Dentro da sua gitlab dashboard, na sessão de jobs, deve existir um output como esse:

Service [api-dev] revision [sua-api-00000-pez] has been deployed and is serving 100 percent of traffic.
Service URL: https://sua-api-hash-uc.a.run.app

Se tudo deu certo, utilizando curl para interagir com o endpoint health:

curl https://sua-api-hash-uc.a.run.app/health
{"health_status":{"status":"healthy","message":"API is working fine"}}%

Repositório da api aqui.

Espero que, na eventualidade de você se encontrar travado com um procedimento como esse, ou parecido, essa postagem te ajuda a desimpedir (:

Deploying Redis on GCP with Helm, Kubernetes, and Pulumi

Rodrigo Burgos — Tue, 27 May 2025 03:36:16 +0000

In this tutorial, you’ll learn how to deploy a highly available Redis instance on Google Kubernetes Engine (GKE) using Pulumi, Helm, and Kubernetes. We'll cover:

Prerequisites
Setting up GCP & GKE via Pulumi
Deploying Redis with the Bitnami Helm chart
Exposing Redis with a LoadBalancer
Validating the Deployment

1. Prerequisites

Before you begin, make sure you have:

A GCP project with billing enabled.
gcloud CLI installed and authenticated.
Pulumi CLI installed.
A local Kubernetes context (you'll dynamically generate one via Pulumi).
Helm installed.
Go ≥1.18 (or your preferred Pulumi language runtime).

2. Setting up GCP & GKE via Pulumi

Create a new Pulumi project (e.g. pulumi new go), then install the GCP and Kubernetes providers:

go get github.com/pulumi/pulumi-gcp/sdk/v6/go/gcp/container
go get github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes

Next, implement a provider package to:

Provision a GKE cluster
Generate a kubeconfig
Instantiate a Kubernetes provider

// provider/provider.go
package provider

import (
  "fmt"
  "os/exec"
  "strings"

  "github.com/pulumi/pulumi-gcp/sdk/v6/go/gcp/container"
  k8s "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes"
  "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func SetupProvider(ctx *pulumi.Context) (*k8s.Provider, error) {
  // Provision a GKE cluster
  cluster, err := container.NewCluster(ctx, "my-gke-cluster", &container.ClusterArgs{
    Location:         pulumi.String("<YOUR_REGION>"),
    InitialNodeCount: pulumi.Int(1),
    NodeConfig: &container.ClusterNodeConfigArgs{
      MachineType: pulumi.String("e2-standard-2"),
      OauthScopes: pulumi.StringArray{
        pulumi.String("https://www.googleapis.com/auth/cloud-platform"),
      },
    },
  })
  if err != nil {
    return nil, fmt.Errorf("creating GKE cluster: %w", err)
  }

  // Build kubeconfig dynamically
  kubeconfig := pulumi.All(cluster.Name, cluster.Endpoint, cluster.MasterAuth).ApplyT(
    func(args []interface{}) (string, error) {
      name := args[0].(string)
      endpoint := args[1].(string)
      auth := args[2].(container.ClusterMasterAuth)

      // Retrieve a short-lived access token
      out, err := exec.Command("gcloud", "auth", "print-access-token").Output()
      if err != nil {
        return "", fmt.Errorf("gcloud auth: %w", err)
      }
      token := strings.TrimSpace(string(out))

      return fmt.Sprintf(`
apiVersion: v1
kind: Config
clusters:
- name: %s
  cluster:
    certificate-authority-data: %s
    server: https://%s
contexts:
- name: %s
  context:
    cluster: %s
    user: %s
current-context: %s
users:
- name: %s
  user:
    token: %s
`, name, *auth.ClusterCaCertificate, endpoint,
        name, name, name, name, name, token), nil
    }).(pulumi.StringOutput)

  // Create the Pulumi Kubernetes provider
  k8sProvider, err := k8s.NewProvider(ctx, "k8s", &k8s.ProviderArgs{
    Kubeconfig: kubeconfig,
  })
  if err != nil {
    return nil, fmt.Errorf("creating Kubernetes provider: %w", err)
  }

  return k8sProvider, nil
}

Finally, call this from your main.go:

// main.go
package main

import (
  "iac_staging/provider"
  "iac_staging/redis"

  "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
  pulumi.Run(func(ctx *pulumi.Context) error {
    // Setup GKE + Kubernetes provider
    k8sProvider, err := provider.SetupProvider(ctx)
    if err != nil {
      return err
    }

    // Deploy Redis
    if err := redis.Install(ctx, k8sProvider); err != nil {
      return err
    }

    return nil
  })
}

3. Deploying Redis with the Bitnami Helm Chart

Create a redis package that uses Pulumi’s Helm integration to install the Bitnami Redis chart:

// redis/redis.go
package redis

import (
  "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/helm/v3"
  k8s "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes"
  "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func Install(ctx *pulumi.Context, provider *k8s.Provider) error {
  // Replace <YOUR_REDIS_PASSWORD> with a Pulumi secret or config
  redisPassword := pulumi.String("<YOUR_REDIS_PASSWORD>")

  _, err := helm.NewChart(ctx, "redis", helm.ChartArgs{
    Chart:     pulumi.String("redis"),
    Version:   pulumi.String("17.3.11"),
    Namespace: pulumi.String("default"),
    FetchArgs: helm.FetchArgs{
      Repo: pulumi.String("https://charts.bitnami.com/bitnami"),
    },
    Values: pulumi.Map{
      "auth": pulumi.Map{
        "enabled":  pulumi.Bool(true),
        "password": redisPassword,
      },
      "master": pulumi.Map{
        "service": pulumi.Map{
          // Type LoadBalancer will provision a GCP Network LB
          "type": pulumi.String("LoadBalancer"),
        },
        "resources": pulumi.Map{
          "requests": pulumi.Map{
            "cpu":    pulumi.String("100m"),
            "memory": pulumi.String("128Mi"),
          },
          "limits": pulumi.Map{
            "cpu":    pulumi.String("250m"),
            "memory": pulumi.String("256Mi"),
          },
        },
      },
      "replica": pulumi.Map{
        "replicaCount": pulumi.Int(1),
        "resources": pulumi.Map{
          "requests": pulumi.Map{
            "cpu":    pulumi.String("100m"),
            "memory": pulumi.String("128Mi"),
          },
          "limits": pulumi.Map{
            "cpu":    pulumi.String("250m"),
            "memory": pulumi.String("256Mi"),
          },
        },
      },
    },
  }, pulumi.Provider(provider))
  return err
}

4. Exposing Redis with a LoadBalancer

By setting the service.type to LoadBalancer, GKE will automatically provision a GCP Network Load Balancer with an external IP. You can retrieve this IP with:

kubectl --context $(pulumi stack output kubeconfig) \
  get svc redis-master --namespace default -o jsonpath='{.status.loadBalancer.ingress[0].ip}'

Tip: You can also configure a static IP in GCP and annotate the chart to use it by adding:

master:
  service:
    annotations:
      networking.gke.io/load-balancer-type: "External"
    loadBalancerIP: "YOUR_RESERVED_STATIC_IP"

5. Validating the Deployment

Obtain the External IP:

EXTERNAL_IP=$(kubectl get svc redis-master \
  --namespace default \
  -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo "Redis external IP: $EXTERNAL_IP"

Test Connectivity with redis-cli:

redis-cli -h $EXTERNAL_IP -a $REDIS_PASSWORD PING
# Expected: PONG

Conclusion
You’ve now:

Provisioned a GKE cluster on GCP with Pulumi.
Installed Redis using the Bitnami Helm chart via Pulumi.
Exposed Redis publicly (or privately, if you choose a different service.type).
Validated the deployment with redis-cli.

This approach lets you manage your infrastructure and application in a single Pulumi program, ensuring repeatable, version-controlled deployments.

Feel free to drop any questions or improvements in the comments!

Building a Rust API to retrieve Your DEV.to Posts

Rodrigo Burgos — Tue, 22 Apr 2025 11:55:38 +0000

Have you ever wanted a lightweight, self‑hosted API that serves your own DEV.to articles in JSON?

In this tutorial I’ll walk you through the core ideas behind a small Rust service that fetches your posts directly from the DEV.to /api/articles/me/all endpoint and exposes them at GET /posts on localhost:3030.

First, we use Warp as our web server and routing library. Warp’s filter system makes it trivial to mount a route, handle query parameters, and return JSON. In main.rs we set up a single filter for warp::path("posts"), combine it with warp::get(), and dispatch to a handler function.

The handler uses a “use case” module powered by Reqwest and Serde. When you ping /posts, the service reads your API_KEY (from a .env file), builds an HTTP client, and issues a GET to https://dev.to/api/articles/me/all with the required headers:

• api-key:
• Accept: application/vnd.forem.api-v1+json
• User-Agent: reqwest

The JSON response is deserialized into a Vec struct, defined with #[serde(rename_all = "snake_case")] and fields matching the API (id, title, description, slug, url, tag_list, etc). Finally, we wrap that vector in a GetApiCallOutputType and return it as JSON via Warp.

Because it’s pure Rust, this service is fast, statically linked, and easy to containerize. You can build it in release mode (cargo build --release) or package it in Docker with a multi‑stage Dockerfile. Once running, everything is accessible on port 3030.

Next steps could include adding pagination support, in‑memory or Redis caching, request throttling, or even basic auth to protect your local endpoint. Give it a spin, tweak it to your needs, and let me know how you extend it!

You check the git repository Here

Access AWS ElastiCache from Localhost Using a Bastion Host and SSM

Rodrigo Burgos — Wed, 09 Apr 2025 05:42:20 +0000

When managing cloud infrastructure, you often want to securely access services like Redis (ElastiCache) that live in private subnets. Direct access from your machine is usually not allowed — and that’s a good thing. But what if you still want to test or inspect your Redis cluster?

One approach is to set up a bastion host with SSM (Session Manager) and use it to tunnel requests to ElastiCache from your localhost — without exposing anything to the public internet.

Let’s break down how to do it using Terraform and the AWS CLI.

Infrastructure Overview

Here’s what we’re setting up:

A bastion EC2 instance in a public subnet.
A Redis ElastiCache cluster in private subnets.
Security Groups allowing just enough access to get the job done.
SSM Session Manager to connect to the bastion host without SSH or public key management.

Terraform Resources

EC2 Bastion host

resource "aws_instance" "bastion_host" {
  ami                         = "ami-02f3f602d23f1659d"
  instance_type               = "t3.micro"
  subnet_id                   = var.subnet_id_a
  associate_public_ip_address = true
  key_name                    = "your_key"
  iam_instance_profile        = aws_iam_instance_profile.ssm_profile.name
  vpc_security_group_ids      = [aws_security_group.bastion_sg.id]

  root_block_device {
    volume_size = 10
  }

  user_data = file("user_data.sh")
}

IAM Role for SSM

resource "aws_iam_role" "ssm_role" {
  name = "ssm-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [{
      Effect = "Allow",
      Principal = {
        Service = "ec2.amazonaws.com"
      },
      Action = "sts:AssumeRole"
    }]
  })

  managed_policy_arns = [
    "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
  ]
}

resource "aws_iam_instance_profile" "ssm_profile" {
  name = "ssm-profile"
  role = aws_iam_role.ssm_role.name
}

Bastion Host Security Group Allow only egress to required ports (Redis, HTTP, HTTPS):

resource "aws_security_group" "bastion_sg" {
  name   = "bastion-sg"
  vpc_id = var.vpc_id

  egress {
    from_port   = 6379
    to_port     = 6379
    protocol    = "tcp"
    cidr_blocks = ["10.0.0.0/16"] # restrict to your VPC range
  }

  egress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

ElastiCache Redis Cluster

resource "aws_elasticache_cluster" "redis" {
  cluster_id           = "redis-dev"
  engine               = "redis"
  node_type            = "cache.t2.micro"
  num_cache_nodes      = 1
  port                 = 6379
  subnet_group_name    = aws_elasticache_subnet_group.redis_subnets.name
  security_group_ids   = [aws_security_group.redis_sg.id]
}

Redis Security Group

Allow ingress only from the bastion host:

resource "aws_security_group" "redis_sg" {
  name   = "redis-sg"
  vpc_id = var.vpc_id

  ingress {
    from_port       = 6379
    to_port         = 6379
    protocol        = "tcp"
    security_groups = [aws_security_group.bastion_sg.id]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

Accessing Redis from Localhost

Start the SSM session On your local machine, use the AWS CLI to start a session:

aws ssm start-session \
  --target i-xxxxxxxxxxxxxxxxx \
  --profile your-aws-profile \
  --region us-east-1

Connect to Redis from the bastion

redis-cli -h redis-dev.xxxxxx.use1.cache.amazonaws.com -p 6379

You should see:

redis-dev.xxxxxx.use1.cache.amazonaws.com:6379> ping
PONG

Nice — Redis is alive 🎉

Bonus: Security Best Practices

✅ Never use 0.0.0.0/0 unless you’re testing — always restrict IPs or CIDRs.

✅ Use private subnets for ElastiCache.

✅ Limit egress/ingress traffic via security groups.

✅ Prefer SSM over SSH for bastion access (no exposed ports).

✅ Remove public IPs once everything is connected through VPC peering or VPNs.

Deploying Redis and an ECS Service with websocket on AWS with Terraform

Rodrigo Burgos — Wed, 02 Apr 2025 20:55:58 +0000

Amazon Web Services (AWS) provides a robust infrastructure for deploying scalable applications. In this tutorial, we will walk through setting up an Amazon ElastiCache Redis cluster and an ECS (Fargate) service using Terraform.

This setup includes:

A Redis cluster for caching.
An ECS service running an API container.
Security groups to control network access.
CloudWatch logging for monitoring.

Prerequisites

Before you begin, ensure you have the following installed:

Terraform (latest version)
AWS CLI (configured with credentials)
A VPC with available subnets
An ECR repository for your application image

1. Configuring the Redis Cluster

First, create an ElastiCache parameter group to customize Redis settings:

resource "aws_elasticache_parameter_group" "custom_redis_parameter_group_dev" {
  name   = "custom-redis-parameter-group-dev"
  family = "redis7"

  parameter {
    name  = "maxmemory-policy"
    value = "allkeys-lru"
  }

  parameter {
    name  = "timeout"
    value = "3600"
  }
}

Now, define a subnet group for Redis:

resource "aws_elasticache_subnet_group" "redis_public_subnet_group_dev" {
  name       = "redis-public-subnet-group-dev"
  subnet_ids = [var.subnet_id_a, var.subnet_id_b]

  tags = {
    Name = "redis-public-subnet-group-dev"
  }
}

Next, create the Redis cluster:

resource "aws_elasticache_cluster" "redis_cluster_dev" {
  cluster_id           = "redis-cluster-dev"
  engine               = "redis"
  node_type            = "cache.t2.micro"
  num_cache_nodes      = 1
  parameter_group_name = aws_elasticache_parameter_group.custom_redis_parameter_group_dev.name
  port                 = 6379
  security_group_ids   = [aws_security_group.redis_dev_sg.id]
  subnet_group_name    = aws_elasticache_subnet_group.redis_public_subnet_group_dev.name
}

2. Configuring the ECS Service

Define the ECS Task Definition:

resource "aws_ecs_task_definition" "game_api_task" {
  family                   = "game-api-task-dev"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = "256"
  memory                   = "512"
  execution_role_arn       = aws_iam_role.ecs_task_execution_role.arn
  task_role_arn            = aws_iam_role.ecs_task_role.arn

  container_definitions = jsonencode([
    {
      name      = "game_api_container_dev"
      image     = "${aws_ecr_repository.game_api_ecr_dev.repository_url}:latest"
      cpu       = 256
      memory    = 512
      essential = true

      portMappings = [
        {
          containerPort = 3000
          hostPort      = 3000
          protocol      = "tcp"
        }
      ]

      logConfiguration = {
        logDriver = "awslogs"
        options = {
          awslogs-group         = aws_cloudwatch_log_group.game_api_dev_log_group.name
          awslogs-region        = "us-east-1"
          awslogs-stream-prefix = "ecs"
        }
      }
    }
  ])
}

Create a CloudWatch log group for logs:

resource "aws_cloudwatch_log_group" "game_api_dev_log_group" {
  name              = "/ecs/game-api-task-dev"
  retention_in_days = 7
}

3. Configuring Security Groups

Define the security group for the API Load Balancer:

resource "aws_security_group" "api_load_balancer_dev_sg" {
  name        = "api-sg-dev"
  description = "Allow HTTP traffic"
  vpc_id      = var.vpc_id

  ingress {
    description = "Allow HTTP traffic"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "Allow HTTPS traffic"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "Allow API traffic"
    from_port   = 3000
    to_port     = 3000
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    description = "Allow all outbound traffic"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

Define the security group for Redis:

resource "aws_security_group" "redis_dev_sg" {
  name        = "redis-sg-dev"
  description = "Security group for Redis cluster"
  vpc_id      = var.vpc_id

  ingress {
    description = "Allow Redis access from ECS"
    from_port   = 6379
    to_port     = 6379
    protocol    = "tcp"
  }

Implementing a WebSocket Gateway in NestJS

In real-time applications, WebSockets enable bidirectional communication between clients and servers. NestJS provides a built-in WebSocket module to facilitate this. Below, we define a WebSocket gateway using the @nestjs/websockets package.

Setting Up the WebSocket Gateway

The MatchGateway class listens for specific messages and manages client connections.

import {
  SubscribeMessage,
  WebSocketGateway,
  WebSocketServer,
  MessageBody,
  ConnectedSocket
} from "@nestjs/websockets";
import { Server, Socket } from "socket.io";
import { PlayerConnectionDto } from "./dto/player.dto";
import { SocketMessagesEnum } from "@/domain/enum/socket-messages";
import { AddPlayerToRoundUseCase } from "@/use-cases/player/add-player-to-round";

@WebSocketGateway({
  cors: {
    origin: "*",
    methods: ["GET", "POST"]
  },
  transports: ["websocket"]
})
export class MatchGateway {
  @WebSocketServer() server: Server;

  constructor(
    private readonly addPlayerToRoundUseCase: AddPlayerToRoundUseCase
  ) {}

  handleConnection(client: Socket): void {
    console.log("Client connected:", client.id);
  }

  @SubscribeMessage(SocketMessagesEnum.PLAYER_CONNECTION)
  async handlePlayerOn(
    @MessageBody() player: PlayerConnectionDto,
    @ConnectedSocket() client: Socket
  ): Promise<void> {
    try {
      const playerData = { id: player.id };
      client.emit(SocketMessagesEnum.PLAYER_CONNECTION, playerData);
    } catch (error) {
      console.error("Error in WebSocket handler:", error);
      client.emit(SocketMessagesEnum.ERROR, {
        message: `Error processing player connection: ${error.message}`
      });
    }
  }

  @SubscribeMessage(SocketMessagesEnum.CHART_DATA)
  async handleChartData(): Promise<void> {
    try {
      // Handle real-time chart data
    } catch (error) {
      console.error("Error in chart data handler:", error);
    }
  }
}

Setting Up the WebSocket Adapter

To ensure the WebSocket server runs smoothly within the NestJS application, an adapter is needed. This can be added in the bootstrap function:

import { IoAdapter } from "@nestjs/platform-socket.io";
import { NestFactory } from "@nestjs/core";
import { AppModule } from "./modules/app.module";

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  app.useWebSocketAdapter(new IoAdapter(app));
  app.enableCors({ origin: "*", methods: ["GET", "POST"] });

  const port = process.env.PORT || 3000;
  await app.listen(port);
  console.log(`🚀 Server running on port ${port}`);
}

bootstrap();

Automating Deployment with GitHub Actions

To automate deployments, create a GitHub Actions workflow (.github/workflows/deploy.yml) with the following:

name: Deploy API to ECS

on:
  push:
    branches:
      - dev

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout the repository
        uses: actions/checkout@v3

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v3
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-east-1"

      - name: Login to Amazon ECR
        uses: aws-actions/amazon-ecr-login@v1

      - name: Build, tag, and push the Docker image to Amazon ECR
        env:
          IMAGE_TAG: ${{ github.sha }}
        run: |
          docker build -t $ECR_REPOSITORY:$IMAGE_TAG .
          docker push $ECR_REPOSITORY:$IMAGE_TAG

      - name: Deploy to ECS
        run: |
          aws ecs update-service --cluster $ECS_CLUSTER --service $ECS_SERVICE --force-new-deployment

Why Deploying NestJS on AWS Lambda with Docker is a Nightmare

Rodrigo Burgos — Tue, 25 Feb 2025 00:03:32 +0000

🚨Do not even try to deploy your NestJS/Javascript app's on lambda through docker🚨

1. AWS Lambda Was Not Designed for Heavy Docker Images

✅ What Lambda was designed for:

Small, lightweight Node.js functions that start up in milliseconds.
Short-lived execution with minimal dependencies.

❌ Why Docker breaks this:

Even the most optimized NestJS Docker image is HUGE (~500MB+).
Lambda pulls the container on every cold start, which can take seconds to minutes.
Containerized apps have a higher startup time, defeating the purpose of Lambda's serverless scalability.

2. AWS Lambda + Prisma is a Disaster

✅ How a normal NestJS server works:

Runs continuously and stores dependencies on nod_modules.

❌ Why Lambda + Docker makes cold starts worse:

Every cold start initializes a fresh Docker container (~5–15s startup).
Lambda must pull the container from AWS ECR, which adds even more latency.
NestJS bootstrapping is slow because of its dependency injection system.
If inside a VPC, cold starts can exceed 10–30 seconds (!).

Special point:

The directory where .prisma package goes is kinda unpredictable. For some wierd reason that's the place where my package goes after running prisma generate inside the Dockerfile:

./node_modules/.pnpm/@prisma+client@6.2.1_prisma@6.2.1/node_modules/@prisma/client /app/node_modules/.prisma/client

And i was only able to perceive this because i was echoing the directory where the library went through github actions.

3. AWS Lambda’s Container Runtime Has Many Restrictions
✅ How a regular NestJS Docker container works:

You control everything: OS, networking, environment.

❌ Why Lambda makes this impossible:

AWS Lambda’s container runtime restricts system calls (you can't run background tasks).
Filesystem is read-only, so Prisma’s binary caching breaks. Lambda has no long-lived storage unless using EFS (which adds latency and complexity).
Can’t run extra processes inside the container (e.g., you can't run a worker process in the background).

4. Debugging is a Nightmare

✅ Normal Docker deployments (ECS, Kubernetes) have:

Persistent logs
SSH access to debug issues
❌ Why AWS Lambda with Docker is a black box:

No real-time debugging (you can only check CloudWatch logs after execution).
Lambda crashes without logs (Runtime.ExitError with no explanation).
You need to login into ECS and then access the container, to get the logs inside of it.

TL;DR:

Cold starts kill performance.
Prisma and databases don’t work well in Lambda.
Docker images are too big for Lambda’s fast-scaling model.
Debugging is painful.

If you have huge NestJS app (basically everything is huge on NestJS app's), go for ALB or break it (if possible) in microsservices. Avoid using Prisma or any huge library, you also can't use some cryptography libraries such as bcrypt.

Automate Testing and Release of npm Packages with GitHub Actions

Rodrigo Burgos — Mon, 17 Feb 2025 12:16:52 +0000

Still some working to be done, such as release control within github itself. Maybe i'll update this post later with these features.

Step 1: Creating an npm Authentication Token

To publish your package from GitHub Actions, you need an authentication token:

Go to npmjs.com.

Click Generate New Token with at least Publish access.

Copy the generated token.

In your GitHub repository, go to Settings > Secrets and variables > Actions.

Click New repository secret, name it NPM_TOKEN, and paste the token.

Step 2: Setting Up GitHub Actions Workflow

Create a .github/workflows/test-and-publish.yml file in your repository with the following content:

name: Test and Publish

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout repository
      uses: actions/checkout@v3

    - name: Set up Node.js
      uses: actions/setup-node@v3
      with:
        node-version: '20'

    - name: Install dependencies
      run: yarn install --frozen-lockfile

    - name: Run tests
      run: yarn test

  publish:
    needs: test
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main' && github.event_name == 'push'

    steps:
    - name: Checkout repository
      uses: actions/checkout@v3

    - name: Set up Node.js
      uses: actions/setup-node@v3
      with:
        node-version: '20'
        registry-url: 'https://registry.npmjs.org/'

    - name: Install dependencies
      run: yarn install --frozen-lockfile

    - name: Publish to npm
      run: yarn publish --access public
      env:
        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

Explanation of Workflow

Runs on every push and pull request to main.
Checks out the repository.
Sets up Node.js version 20.
Installs dependencies using yarn install --frozen-lockfile to ensure dependency consistency.
Runs tests using yarn test.
Publish Job
Runs only if tests pass and the event is a push to main.
Ensures a clean checkout of the repository.
Sets up Node.js with the npm registry.
Installs dependencies.
Publishes the package using yarn publish --access public.

Committing and Running the Workflow

Commit and push the .github/workflows/test-and-publish.yml file to your repository. Once pushed:
GitHub Actions will trigger a test run.
If the tests pass and the branch is main, the package will be published to npm automatically.

Automating AWS API Gateway Deployment with GitHub Actions and OpenAPI 3.0

Rodrigo Burgos — Fri, 14 Feb 2025 18:57:32 +0000

In this guide, we'll walk through setting up a CI/CD pipeline using GitHub Actions to deploy an AWS API Gateway with an OpenAPI 3.0 specification. This automation will streamline updates to your API Gateway whenever changes are pushed to the dev branch.

Prerequisites

Before proceeding, ensure you have:
An AWS account with API Gateway permissions.
An API Gateway REST API already created.
An OpenAPI 3.0 definition file (openapi.json) in your repository.
GitHub Secrets configured with:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_REGION

GitHub Actions Workflow

Create a new workflow file in your repository at .github/workflows/deploy-api-gateway.yml and add the following configuration:

name: Deploy API Gateway

on:
  push:
    branches:
      - dev
  workflow_dispatch:

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v3
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ secrets.AWS_REGION }}

      - name: Update API Gateway with OpenAPI definition
        run: |
          aws apigateway put-rest-api \
            --rest-api-id ${{ secrets.API_GATEWAY_ID }} \
            --body "$(cat openapi.json | base64 --wrap=0)"

      - name: Deploy API Gateway
        run: |
          aws apigateway create-deployment \
            --rest-api-id ${{ secrets.API_GATEWAY_ID }} \
            --stage-name dev

Explanation of Steps

Trigger Conditions: The workflow runs on a push to the dev branch or when manually triggered.
Checkout Repository: Clones the repository so we can access the OpenAPI file.
Configure AWS Credentials: Uses the GitHub Secrets to authenticate with AWS.
Update API Gateway: Uploads the latest OpenAPI 3.0 definition to API Gateway.
Deploy API Gateway: Creates a new deployment in the dev stage.

Benefits of Automation

Ensures API Gateway stays in sync with the OpenAPI definition.
Eliminates manual updates, reducing errors.
Provides a clear version history through GitHub Actions logs.
By implementing this workflow, your API updates will be automatically deployed, allowing for a more efficient development process.

AWS Systems Manager (SSM) to perform Prisma operations on a closed RDS instance on github actions

Rodrigo Burgos — Wed, 12 Feb 2025 19:12:05 +0000

Prerequisites

AWS RDS Instance: Your RDS instance must be configured to accept connections from localhost (when the EC2 instance is used as a bastion to connect).
SSM Agent: Ensure that your EC2 instance (acting as the bastion host) has the SSM agent installed and running.
IAM Roles: The IAM role associated with your EC2 instance must have the necessary permissions to use AWS Systems Manager (SSM) and access RDS resources.
VPC Security Group: Your EC2 instance should have the right security group and routing configured to connect to the RDS instance.

Steps

Prepare your environment with terraform

Make sure your EC2 instance has the required IAM role attached with the necessary permissions:

resource "aws_iam_role" "role_acesso_ssm" {
  assume_role_policy    = "{\"Statement\":[{\"Action\":\"sts:AssumeRole\",\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"ec2.amazonaws.com\"}}],\"Version\":\"2012-10-17\"}"
  managed_policy_arns   = [
    "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess",
    "arn:aws:iam::aws:policy/AmazonS3FullAccess",
    "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
  ]
  name                  = "role-acesso-ssm"
}

This role ensures the EC2 instance can perform operations on SSM and connect to the necessary resources.

Enable Port Forwarding with SSM on github actions

Once your EC2 instance has the necessary IAM roles and SSM agent installed, you'll set up port forwarding using AWS Systems Manager. Port forwarding allows you to connect to a closed RDS instance through the bastion host without opening its security group.

Start an SSM Session to forward the port (e.g., port 5432 for PostgreSQL) from the bastion host to the RDS instance:

INSTANCE_ID=$(aws ec2 describe-instances --filters "Name=tag:Name,Values=my-bastion-host" --query "Reservations[0].Instances[0].InstanceId" --output text)

aws ssm start-session --target $INSTANCE_ID \
  --document-name AWS-StartPortForwardingSessionToRemoteHost \
  --parameters '{"host":["my-rds-instance.rds.amazonaws.com"],"portNumber":["5432"],"localPortNumber":["5432"]}'

This command will establish a secure connection between your EC2 instance and RDS, and allow you to connect to the database locally on your machine via port 5432.

Setting Up Environment Variables

You’ll need environment variables in your github secrets to securely connect to your RDS instance using Prisma. These should include your database credentials, which are best stored in AWS Secrets Manager or as environment variables.

For example:

"postgresql://username:password@localhost:5432/my_database"

Perform Prisma Operations

Now that you have port forwarding in place, you can interact with the closed RDS instance using Prisma from your dockerfile.

# Generate Prisma Client
RUN pnpm prisma generate

Important Notes:

Security: Ensure your IAM roles and permissions are securely configured to avoid unnecessary exposure to sensitive resources. Port Forwarding: If the RDS instance is closed, port forwarding via SSM is a great way to establish a secure tunnel without exposing the database publicly.