DEV Community: Marcos Henrique

in case of fire put your heap on a cage - how we reduced our mem usage

Marcos Henrique — Fri, 06 Mar 2026 23:40:23 +0000

A few weeks ago, I dropped a message in our team Slack that made everyone stop what they were doing:

"The work we have made on axios interceptors refactor + removing in memory cache + v8 tuning seems to improved mem usage A LOT"

Some graphs came with it. One of those graphs where you don't need to read the y-axis to understand what happened 🙇‍♂️, the line simply drops and stays down. AWS confirmed it at the same timestamp.

This is the story of how we got there...

Spoiler: Datadog with its heap snapshot is the chef's kiss

Preface (⚠️ DD&ECS&CDK newbies only)

Just in case you never configured Datadog Tracing and created an ECS Cluster on Fargate fashion, with CDK obviously, if you already did it, jump to the next "Section"...

1. Creating the ECS Cluster

import * as ecs from 'aws-cdk-lib/aws-ecs';
  import * as ec2 from 'aws-cdk-lib/aws-ec2';

  const vpc = ec2.Vpc.fromLookup(this, 'Vpc', { tags: { Name: 'default-vpc' } });

  const cluster = new ecs.Cluster(this, 'MainCluster', {
      clusterName: 'main-cluster',
      vpc,
      // Fargate doesn't need capacity providers or EC2 instances
      containerInsights: true,  // enables CloudWatch Container Insights
  });

1.1 Service Connect (just in case you don't want to pay extra money on egress/ingress)

const cluster = new ecs.Cluster(this, 'MainCluster', {
      clusterName: 'main-cluster',
      vpc,
      containerInsights: true,
      defaultCloudMapNamespace: {
          name: 'playground.local',                        // internal DNS namespace
          type: cloudmap.NamespaceType.DNS_PRIVATE,    // private to the VPC
          vpc,
      },
  });

2. Task definition + Datadog agent sidecar

  const taskDef = new ecs.TaskDefinition(this, 'TaskDef', {
    compatibility: ecs.Compatibility.FARGATE,
    cpu: '1024',
    memoryMiB: '2048',
    networkMode: ecs.NetworkMode.AWS_VPC,
  });

  // Datadog agent sidecar
  taskDef.addContainer('datadog-agent', {
    image: ecs.ContainerImage.fromRegistry('public.ecr.aws/datadog/agent:latest'),
    essential: true,
    cpu: 256,
    memoryReservationMiB: 512,
    environment: {
      DD_API_KEY: datadogApiKey,
      DD_SITE: 'datadoghq.com',
      ECS_FARGATE: 'true',
      DD_APM_ENABLED: 'true',
      DD_APM_NON_LOCAL_TRAFFIC: 'true',
      DD_PROFILING_ENABLED: 'true',      // enables Memory Leaks tab
      DD_RUNTIME_METRICS_ENABLED: 'true',
    },
    portMappings: [
      { containerPort: 8125, protocol: ecs.Protocol.UDP },
      { containerPort: 8126, protocol: ecs.Protocol.TCP },
    ],
  });

3. App container wired to the agent

  taskDef.addContainer('my-api', {
    image: ecs.ContainerImage.fromRegistry('my-ecr-repo/api:latest'),
    essential: true,
    cpu: 768,
    memoryReservationMiB: 1536,
    environment: {
      DD_SERVICE: 'my-api',
      DD_ENV: 'production',
      DD_VERSION: '1.0',
      DD_TRACE_AGENT_URL: 'http://localhost:8126', // points to sidecar
      DD_PROFILING_ENABLED: 'true',
      DD_RUNTIME_METRICS_ENABLED: 'true',
    },
    dockerLabels: {
      'com.datadoghq.tags.service': 'my-api',
      'com.datadoghq.tags.env': 'production',
      'com.datadoghq.tags.version': '1.0',
    },
  });

4. Node.js app — init tracer before anything else

  // tracer.ts — must be the first import in main.ts
  import tracer from 'dd-trace';

  tracer.init({
    profiling: true,       // heap snapshots → Memory Leaks tab
    runtimeMetrics: true,  // Node.js live heap graph
    logInjection: true,    // correlates logs with traces
  });

  // main.ts
  import './tracer';
  import { NestFactory } from '@nestjs/core';
  // ...

///usage...
import tracer from 'dd-trace';                                                                                                                 

  async function fooFunc(memberId: string) {                                                                                                  
    const span = tracer.startSpan('action.subAction', {                                                                                               
      tags: { 'action.id': actionId },                                                                                                           
    });                                                                                                                                          

    try {
      await foo.bar(actionId);
      span.finish();
    } catch (err) {
      span.setTag('error', err);
      span.finish();
      throw err;
    }
  }
///...

How it will look on DataDog

The Dashboard Doesn't Lie

Like most memory issues, ours wasn't dramatic. There was no crash, no alert, no customer ticket... Just a Datadog APM dashboard showing memory usage slowly creeping upward between deployments 👻.
The kind of thing that's easy to rationalise:

maybe it's traffic growth, maybe it's a new feature, maybe it's just Node being Node (since our traffic was growing)

Except it kept going. And at some point, we started asking:

What if it's us?

Spoiler: it was us

The Axios Interceptor Trap

We use axios-retry across a lot of our outbound integrations, so the pattern across all of them looked like this:

async createMeeting(data: MeetingData) {
  axiosRetry(this.http.axiosRef, {
    retries: 3,
    retryDelay: axiosRetry.exponentialDelay,
    retryCondition: axiosRetry.isRetryableError,
  });

  // ... the actual HTTP request
}

Looks fine, right? You're just configuring retry behaviour before making a request. Completely reasonable.

Except axiosRetry doesn't just configure behaviour. It registers interceptors.

Every call to axiosRetry(instance, options) calls instance.interceptors.request.use(...) and instance.interceptors.response.use(...) under the hood. Those arrays in Axios grow with every call. They never get cleaned up. Every function reference that gets pushed there lives on the heap for the lifetime of the process.

Therefore, what was actually happening: every incoming request to our API triggered outbound calls to third-party services. Every one of those outbound calls registered a new pair of retry interceptors. After thousands of requests, I had thousands of interceptors sitting quietly in memory... all alive 🧟‍♂️..., all holding references, all doing redundant work.

The fix is almost embarrassingly 😳 simple: move the registration to the constructor, so it happens exactly once per service instance

constructor(private http: HttpService) {
  axiosRetry(this.http.axiosRef, {
    retries: 3,
    retryDelay: axiosRetry.exponentialDelay,
    retryCondition: axiosRetry.isRetryableError,
  });
}

We applied this across every service that was doing it wrong, yeah, that was a lot of places. And every single one of them was leaking 🤯

This was almost certainly the biggest contributor to what we saw on the graphs

What Lives in Your Heap Shouldn't Have to 🧟‍♂️

While investigating the interceptor issue, we also looked at our MemoryCacheService. It was using a plain JavaScript Map to cache lab scheduling slot data, a totally normal pattern for reducing external API calls.

The issue isn't that caching is bad. It's that in-process heap memory is a shared, finite resource. Lab scheduling data cached indefinitely in a Map competes with the V8 heap for space. As the cache grows, so does GC pressure. As GC pressure grows, pause times get longer, and memory spikes become more visible.

We replaced the in-memory Map with a dedicated Redis (you should use Valkey, but we were using the boomer redis 🥱) connection (DB 5, isolated from our primary data). SCAN-based pattern deletes, PX TTL, JSON serialise/deserialise. The interface to all consumers stayed exactly the same, they never knew the difference. But the data no longer lives in the Node.js heap.

This is a meaningful architectural shift: anything that doesn't need to be in-process shouldn't be. Redis is fast enough for this use case, so cache growth no longer means heap growth.

V8 Knows More Than I Was Telling It

The third piece was more subtle. I added the --optimize-for-size flag to both our API and worker startup commands:

"start:prod": "node --optimize-for-size -r dd-trace/init dist/src/main"

This flag tells V8 to bias toward a smaller memory footprint over raw JIT throughput. In a containerized, multi-instance deployment — where memory is often the binding constraint and CPU headroom is available — that's usually the right trade-off.

The inspiration came from reading Platformatic's deep dive, "We cut Node.js' Memory in half".
They explored V8's pointer compression feature — a mechanism that reduces each heap pointer from 64 bits to 32 bits, effectively halving how much space every JavaScript object consumes in memory. Chrome has used it since 2020. Node.js is only now catching up, and Platformatic did the work to make it accessible.

--optimize-for-size is a lighter version of the same philosophy: tell V8 to prefer compact representations over aggressive JIT optimisations. The JIT compiles more compact bytecode, uses smaller internal structures, and is generally more aggressive about returning memory to the operating system. For a server handling many concurrent short-lived requests rather than long-running CPU-heavy computations, the trade-off consistently lands in your favour.

What the Graphs Showed

The drop was visible and immediate. Not a gradual downward trend — an actual drop at the deployment timestamp, confirmed independently by AWS metrics. The kind of result that makes the whole investigation feel worth it.

One of the engineers on the team put it well in the thread that followed:

"You guys should let the wider dev team know — how you found it and also the reasoning behind. The rest of the team can also pick up the source and the worker, too… I imagine we'll get benefits there too."

That's exactly why I'm writing this.

But Wait!!!! There's a Better Standard Now: `node-caged`

Everything I described above works today, without any tooling changes. But I'd be doing you a disservice if I didn't point to where this is all heading.

The Platformatic team published a Docker image called node-caged that enables V8 pointer compression in Node.js with a one-line Dockerfile swap:

# Before
FROM node:25-bookworm-slim

# After
FROM platformatic/node-caged:25-slim

No code changes. No configuration. Same Node.js APIs, same behaviour, just with every heap pointer compressed from 8 bytes to 4 bytes.

Why does that matter? Because every JavaScript object on the V8 heap contains multiple internal pointers: to its hidden class (shape), to where its properties are stored, to string values, to prototype chains. All of those get cut in half. For real-world applications with mixed workloads — I/O, JSON parsing, middleware chains, database queries — Platformatic's benchmarks showed a 50% reduction in heap usage with only 2.5% average latency overhead.** And counterintuitively, p99 and max latency actually improved by 7-8%, because a smaller heap means the garbage collector has less to scan and fewer long pauses.

The reason this hasn't been the default in Node.js until now is a technical constraint called the "memory cage" — pointer compression originally forced the entire process (main thread and all workers) to share a single 4GB address space. That's fine for Chrome, where each tab runs in its own process. Not fine for Node.js, where workers share a process. Cloudflare sponsored Igalia to introduce IsolateGroups into V8, which gives each isolate its own 4GB cage. That work landed in Node.js 25, and node-caged ships it as a ready-to-use Docker image.

Should you try it? Yes, with one check first. Native addons built on the legacy NAN (Native Abstractions for Node.js) are incompatible, because pointer compression changes V8's internal object layout. Run:

npm ls nan

If nothing shows up, you're in the clear. Most popular native packages have already migrated to Node-API (sharp, bcrypt, canvas, sqlite3, bufferutil) — the only notable holdout is nodegit.

If your dependency tree is clean, the path forward is:

Swap the Docker base image in staging
Watch your memory usage drop
Confirm your p99 stays within SLO (it probably improves)
Ship it

I haven't validated this in our own staging environment yet, but it's next on the list. The gains I got from fixing interceptors, moving to Redis, and adding --optimize-for-size were real and visible. Pointer compression would multiply them.

The original post from Matteo Collina at Platformatic explains the mechanism in detail: We cut Node.js' Memory in half. Worth a full read. The repo is at github.com/platformatic/node-caged.

Debugging Is an Art 👩‍🎨🎨 — and That's the Real Lesson

There's a post I keep coming back to: "The Real Skill in Programming Is Debugging — Everything Else Is Copy-Paste". The argument is that as you grow as an engineer, you spend less time writing new code and more time doing something harder — understanding why a system isn't behaving the way you think it should. Writing code can be learned from docs and examples. Architecture is, in the author's words, "just a high-level copy-paste." But debugging? That's pure problem solving. It's the part that can't easily be replaced, because it requires holding a mental model of a live system and interrogating it.

This investigation felt exactly like that. Nothing in the codebase was wrong in isolation. Every single service calling axiosRetry inside a method was doing something locally reasonable — setting up retry behavior before a request. The problem only existed at the intersection of how Axios works internally, how many times those methods get called over time, and how V8 manages heap references. You can't find that by reading one file. You find it by asking: what does this code actually do to the system over thousands of calls?

That's the art of it. And it's worth naming explicitly, because the fix itself — moving a function call to a constructor — takes about ten seconds. The hours went into understanding why it mattered.

With AI now generating code faster than ever, I think this dynamic is only going to sharpen. The ability to produce syntax is increasingly cheap. The ability to understand what's broken and why — to trace through logs, follow data flows, form and test hypotheses — that's the scarce skill.
Senior engineers aren't people who write more code. They're the ones who can look at a slowly climbing memory graph and not shrug it off.

The Lessons That Actually Stuck

Know what your libraries do to shared state. axiosRetry doesn't configure your axios instance — it modifies it by pushing to an array. Understanding that difference would have caught this pattern immediately. Any library that hooks into request/response pipelines deserves scrutiny around when and how often it registers those hooks.

In-process cache is heap, full stop. Every object in a Map is an object the GC has to account for. For ephemeral, TTL-bounded data, Redis is usually a better home — especially when you're already running it.

V8 has production-facing flags and most teams ignore them. --optimize-for-size is real. V8 pointer compression via node-caged is real. These are not academic curiosities — they're levers that move production graphs.

Memory leaks don't have to be dramatic to be real. Mine wasn't a smoking gun. It was a pattern repeated across a dozen services, each one individually reasonable, all of them together creating a slow bleed. The investigation just required asking: what does this function actually do?

If you've hit similar issues, especially the Axios interceptor accumulation pattern, which I suspect is far more common than people realize, so, yeah I'd love to compare notes.

The work described in this post landed in PR contributed by André Paris and I.

🍻

Throttling Musings with Api Gateway

Marcos Henrique — Sat, 17 Jan 2026 05:16:59 +0000

Musing n' brewing special coffees

Have been a while since my last writing, working n’ walking the tightrope with some of the craziest foreigners guys with ease, but deep in the gears, like machines in order to arrest Eroom’s Law by delivering industry-wide platform economies of scale via the industrialisation of AI-native scientific data and AI-enabled use cases across the value chain, like on top of petabyte sized data to develop new vaccines, just ordinary stuff, you know? 🤷
And at the same time, trying to explain myself without failing to entertain, yet in the interim, trying to attain a way to explain myself without losing myself while trying to entertain, as a non-native English speaker n’ since contractors produce untrustworthy code (on their own eyes), have you got the drift?
swimming through the void, we hear the word, we lose ourselves, but we find it all.... nvm….musings…

all burned, coffee brewed, may we go to the tech stuff again?

Throttling & Rate Limiting using Api Gateway

I was in charge of coming up with a way to create something like guardrails against attackers, and some of the malicious AI/ML scientists who spin up EC2 P5e/P5en instances as pennies could be cherry-picked off trees, can you imagine? How many dolphins were dying for testing a near-real-time batch inference, just because they like to pet some GPU beasts, yeah…
We have developed this AI Platform (ECS Cluster in a Fargate, always believe in SLS baby)
and they were overusing it, and as a MLOps Engineer (fancy name for Platform Engineering for people who treat production like a science experiment and expect the infrastructure to fix the results, just kiding xD) I was asked to think about Throttling & Rate limting, my proposal was a two layer treatment, but it's hard to sell when they don't want to buy, have you catch my flow? anyway at least the layer 1 was implemented 😅, I'm going to share with you that still following my thoughts:

Layer 1: API Gateway & The Token Bucket

The "I Promise I'm Customer Obsessed" Layer

First up is the Token Bucket in API Gateway. AWS loves this because it’s built-in, which means they can bill you for it without you having to provision a single EC2 instance

Think of the Token Bucket as a vending machine that only accepts "Permission Slips"

The Rate: Every second, AWS drops a set number of slips into the bucket.
The Burst: The bucket has a fixed size. If you don't use your slips, they pile up until the bucket is full
The Reality: When a "passionate" user (read: a botnet from a country you can’t find on a map) hammers your API, they can chug all the slips in the bucket instantly. Once the bucket is empty, API Gateway starts handing out 429 Too Many Requests errors like they’re flyers for a timeshare

It’s great for protecting your account-level limits, but it’s about as surgical as a chainsaw. It doesn't care who is taking the tokens; it just cares that the bucket is empty and it’s time to go home
There is your awaited code example in CDK (sorry, other IaC tools, I've my preferred one):

import * as cdk from 'aws-cdk-lib';
import * as apigateway from 'aws-cdk-lib/aws-apigateway';
import { Construct } from 'constructs';

export class LayerOneStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // 1. The API Gateway: The fancy front door
    const api = new apigateway.RestApi(this, 'MyOverEngineeredApi', {
      restApiName: 'Layer1-Service',
      deployOptions: {
        stageName: 'prod',

        // --- THE TOKEN BUCKET (Throttling) ---
        // Rate: The 'Refiller' (Requests per second)
        // Burst: The 'Bucket Size' (The bank of tokens for spikes)
        throttlingRateLimit: 100, 
        throttlingBurstLimit: 200,

        // --- THE MARGARITA BOWL (Caching) ---
        cachingEnabled: true,
        cacheClusterEnabled: true,
        cacheClusterSize: '0.5', // '0.5' is the smallest and therefore least painful for your bill
        cacheTtl: cdk.Duration.minutes(5),
      },
    });

    // 2. A Resource (The "VIP Section")
    const vips = api.root.addResource('vips');

    // 3. Adding a Method with Integration
    vips.addMethod('GET', new apigateway.MockIntegration({
      integrationResponses: [{
        statusCode: '200',
        responseTemplates: {
          'application/json': '{"message": "Welcome to the club. AWS has already billed you for this sentence."}'
        }
      }],
      passthroughBehavior: apigateway.PassthroughBehavior.NEVER,
    }), {
      methodResponses: [{ statusCode: '200' }],
      // This is the key: tell API Gateway WHAT to cache
      requestParameters: {
        'method.request.querystring.user_id': true
      },
    });
  }
}

Layer 2: Redis/Valkey & The Sliding Window

The "I Don't Trust My Users" Layer

Now, if you actually want to know which of your customers is trying to scrape your entire database at 3:00 AM, you need Layer 2: The Sliding Window, usually implemented in ElastiCache for Valkey (a service whose pricing model I can only describe as "aggressively optimistic")

Unlike the Token Bucket, which is just a simple counter, the Sliding Window is for people with trust issues

Illustrating it for you who likes illustrations:

I know you were thinking, where is the code? There is mate:

import { Valkey } from 'iovalkey'; // The fork that won't ask for your credit card

const valkey = new Valkey({
  host: 'localhost',
  port: 6379,
});

/**
 * THE SLIDING WINDOW LUA SCRIPT
 * Keys: [ratelimit_key]
 * Args: [window_ms, max_requests, current_timestamp]
 */
const slidingWindowScript = `
  local key = KEYS[1]
  local window = tonumber(ARGV[1])
  local limit = tonumber(ARGV[2])
  local now = tonumber(ARGV[3])
  local oldest = now - window

  -- 1. Remove timestamps outside the sliding window
  redis.call('ZREMRANGEBYSCORE', key, 0, oldest)

  -- 2. Check the current count
  local current_count = redis.call('ZCARD', key)

  if current_count < limit then
    -- 3. Add current request and update expiry
    redis.call('ZADD', key, now, now)
    redis.call('PEXPIRE', key, window)
    return {1, current_count + 1} -- [Allowed, New Count]
  else
    return {0, current_count} -- [Denied, Current Count]
  end
`;

async function checkRateLimit(userId: string) {
  const key = `ratelimit:${userId}`;
  const windowMs = 60000; // 1 minute
  const limit = 100;      // 100 requests
  const now = Date.now();

  // Execute the script on Valkey
  const [allowed, count] = await valkey.eval(
    slidingWindowScript,
    1, // Number of keys
    key,
    windowMs,
    limit,
    now
  ) as [number, number];

  if (allowed === 1) {
    console.log(`✅ Request allowed for ${userId}. Count: ${count}/${limit}`);
    return true;
  } else {
    console.log(`❌ 429: Rate limit exceeded for ${userId}. Stop it.`);
    return false;
  }
}

How it works: Instead of a bucket of tokens, you keep a timestamped log of every request a specific user has made.
The Logic: When a request hits, you look at the last 60 seconds relative to right now. You count the logs. If they’ve made 101 requests and the limit is 100, you kick them out.
The Twist: We use Redis/Valkey Sorted Sets for this. We use [ZREMRANGEBYSCORE](https://valkey.io/commands/zremrangebyscore/) to evict "old news" (timestamps older than a minute) and ZCARD to check whether the user is currently being a nuisance

Remember:

Don’t give up and remember that you are writing not only for the community but also for yourself to organise thoughts :) (sodkiewicz, marcin)
thanks for these words, man 🤝

OpenSkills, adding Claude Skills and Superpowers for any agent or IDE

Marcos Henrique — Sat, 20 Dec 2025 03:36:16 +0000

Superpowers is a complete software development workflow for your coding agents, built on top of a set of composable "skills" and some initial instructions that make sure your agent uses them.

I've found this one, and it's pretty nice, it's basically an npm for skills, rules and agents.md cross platform 🤩 a.k.a PRPM

My favourite is:
https://prpm.dev/collections/superpowers

but Marcos, this is only for claude code 😞
NOPE, you can install it like

npm install -g prpm
prpm install collections/superpowers

Then you should use it as shown in this img below.

Under the hood, prpm uses the OpenSkills library and progressive disclosure via AGENTS.md to allow Cursor to load skills as needed or upon request 🥳

I'm that guy whom don't want to add new packages

Oh, I see haha
If you don't want to add a new package, you can do:

Simplified OpenSkills Setup

You can easily make this available globally without introducing an extra package manager, as I currently do this manually with OpenSkills.

Install OpenSkills globally using your preferred package manager (Bun, npm, or yarn):

bun add -g openskills

Install the desired skills (e.g., obra/superpowers) globally. You'll be prompted to select specific skills; pressing Enter installs all of them:

openskills install obra/superpowers --universal --global

Sync Skills to a Project

In the terminal of each project where you want to use these skills, run the sync command: openskills sync

This command updates the project's agents.md file. You can then edit agents.md to remove any skills you don't need, or alternatively, set this up as a Cursor rule. Once synced, the skills will be ready for use in your chats.

Enjoy and happy coding/vibing haha 🍻

Real-Time ALB Log Analysis for Proactive Integration Recovery via Datadog Monitors, Workflows and AWS Lambda

Marcos Henrique — Mon, 08 Dec 2025 03:22:45 +0000

In the realm of the unknown, sometimes we need to reinvent ourselves and make something we call, here in Brazil, "gambiarra," a.k.a. the art of improvisation.

The Problem:

The Drama: Google Calendar events were giving us major side-eye, and we had zero visibility into which users were feeling the pain.
The Cause: Errors were mostly tied to channels that were ghosting us or events that couldn't reach the API.
The Worst Part: Our ELB/ALB hadn't access logs, so we were total in the dark about it

The light of reasoning

Why not add access logs? Since we already use Datadog, I decided to add the ALB/ELB access logs there.

This project was built using Pulumi, but no worries from my end as a CDK lover, let's make it happen. If you haven't worked with Pulumi before, I recommend checking out their documentation. It seems pretty interesting, as you can manage IaC for many providers using the same lib/IaC tool, almost like Terraform, but using Typescript hehe

Adding ALB/ELB Access Logs

What are Access Logs?

The Access Logs are like the security guard's notebook or a paper trail of everything that happens at that intersection.

Every time a car (a request) passes through the Load Balancer, the guard writes down an entry in the notebook. This entry includes all the important "who, what, and when" details.

Each log entry is a line of information that tells a story. It answers questions like:

When did the car arrive? (The exact time)
Who was driving the car? (The user's IP address)
Where were they trying to go? (The web page URL they requested)
Did the car get where it was going? (The status code—like a 200 OK means "Yes, all good!" or a 404 means "No, that page doesn't exist.")
How long did the whole trip take? (How fast the server responded)
Which server did the Load Balancer send them to? (The specific server ID)

Pulumi Code example for a fresh ALB/ELB

Here, we are going to create the ALB/ELB, the target group, and add certificates and listener rules.
our ALB was already created, just showing you guys how to do it using Pulumi

First we need to create the ALB/ELB:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const mainAlb = new awsx.lb.ApplicationLoadBalancer("main-alb", {
    name: "main-alb",
    subnetIds: vpc.publicSubnetIds,
    securityGroups: [securityGroup.id],
    internal: false,
    accessLogs: {
        bucket: lbLogs.id,
        prefix: "access-logs-lb",
        enabled: true,
    }, // the magic happens here xD
    tags: {
        stage: environment,
        managed: "true",
    },
})

Creating the target group:

const webhooksTargetGroup = new aws.lb.TargetGroup("webhooks-tg", {
    port: 80,
    protocol: "HTTP",
    targetType: "ip",
    vpcId: vpc.vpcId,
    healthCheck: {
        enabled: true,
        healthyThreshold: 2,
        unhealthyThreshold: 2,
        interval: 10,
        path: "/api/v1/health-check",
        port: "traffic-port",
    },
});

Http Listener:

// the http listener
const httpsListener = new aws.lb.Listener("httpsListener", {
    loadBalancerArn: mainAlb.loadBalancer.arn,
    port: 443,
    protocol: "HTTPS",
    defaultActions: [{
        type: "fixed-response",
        fixedResponse: {
            contentType: "text/plain",
            statusCode: "404",
            messageBody: "Not Found",
        },
    }],
    sslPolicy: "ELBSecurityPolicy-2016-08",
    certificateArn: myCertificateArn,
});

//additional certificates
new aws.lb.ListenerCertificate("webhooks-attachment", {
    listenerArn: httpsListener.arn,
    certificateArn: webhooksCertificateArn,
});

Listener Rules:

new aws.lb.ListenerRule("rule-webhooks", {
    listenerArn: httpsListener.arn,
    priority: 3,
    actions: [{ type: "forward", targetGroupArn: webhooksTargetGroup.arn }],
    conditions: [{ hostHeader: { values: [`webhooks.${route53ZoneName}`] } }],
});

Pulumi Code Example for an already created ALB/ELB

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// 1. Create S3 bucket for ALB access logs
const albAccessLogsBucket = new aws.s3.Bucket("alb-access-logs", {
    bucket: `alb-access-logs.${environment}.us-east-1.careops`,
    acl: "private",
});

// Bucket policy to allow ELB to write logs
new aws.s3.BucketPolicy("albLogsBucketPolicy", {
    bucket: albAccessLogsBucket.id,
    policy: albAccessLogsBucket.arn.apply((bucketArn) => JSON.stringify({
        Version: "2012-10-17",
        Statement: [
            {
                Effect: "Allow",
                Principal: { AWS: "arn:aws:iam::127311923021:root" }, // ELB service account for us-east-1
                Action: ["s3:PutObject", "s3:GetBucketAcl"],
                Resource: [bucketArn, `${bucketArn}/*`],
            },
        ],
    })),
});

// 2. Enable access logs on ALB
new aws.lb.LoadBalancerAttributes("mainAlbAccessLogs", {
    loadBalancerArn: mainAlb.loadBalancer.arn,
    attributes: [
        { key: "access_logs.s3.enabled", value: "true" },
        { key: "access_logs.s3.bucket", value: albAccessLogsBucket.bucket },
        { key: "access_logs.s3.prefix", value: "main-alb/" },
    ],
});

... so now you should have a bucket that stores your ALB/ELB traffic

ELB publishes a log file for each load balancer node every 5 minutes. Log delivery is eventually consistent. The load balancer can deliver multiple logs for the same period. This usually happens if the site has high traffic.

The file names of the access logs use the following format:
bucket[/prefix]/AWSLogs/aws-account-id/elasticloadbalancing/region/yyyy/mm/dd/aws-account-id_elasticloadbalancing_region_app.load-balancer-id_end-time_ip-address_random-string.log.gz

An example with less variables:
s3://amzn-s3-demo-logging-bucket/logging-prefix/AWSLogs/123456789012/elasticloadbalancing/us-east-2/2022/05/01/123456789012_elasticloadbalancing_us-east-2_app.my-loadbalancer.1234567890abcdef_20220215T2340Z_172.160.001.192_20sg8hgm.log.gz

Forwarding the logs to Datadog

As it is not magic, you need to explicitly send your logs to Datadog, basically by creating a lambda and sending it all

// 3. Deploy Datadog Forwarder Lambda (using AWS SAM/CloudFormation or manual deployment)
// For Pulumi, you can use aws.cloudformation.Stack or deploy the Lambda directly
const datadogForwarder = new aws.lambda.Function("datadogForwarder", {
    runtime: aws.lambda.Runtime.Python39,
    handler: "lambda_function.lambda_handler",
    role: datadogForwarderRole.arn,
    // Use the official Datadog Forwarder layer or package
    code: new pulumi.asset.AssetArchive({
        ".": new pulumi.asset.FileArchive("./datadog-forwarder"),
    }),
    environment: {
        variables: {
            DD_API_KEY_SECRET_ARN: datadogApiKeySecret.arn,
            DD_SITE: "datadoghq.com",
            DD_TAGS: `env:${environment}`,
        },
    },
});

// 4. Configure S3 event notification to trigger Lambda when new logs arrive
new aws.s3.BucketNotification("albLogsNotification", {
    bucket: albAccessLogsBucket.id,
    lambdaFunctions: [{
        lambdaFunctionArn: datadogForwarder.arn,
        events: ["s3:ObjectCreated:*"],
        filterPrefix: "main-alb/AWSLogs/", // ALB logs path pattern
    }],
});

// Grant S3 permission to invoke Lambda
new aws.lambda.Permission("allowS3InvokeLambda", {
    action: "lambda:InvokeFunction",
    function: datadogForwarder.name,
    principal: "s3.amazonaws.com",
    sourceArn: albAccessLogsBucket.arn,
});

After this setup, you will be able to see something like this:

Now our ALB/ELB was spamming error logs, but we didn't have the tea on who was affected. We were fancy, low-key, clueless with our access logs 🫠

The hacky solution

So the idea was to address this lack of observability by adding ALB/ELB Access Logs, but that wasn’t enough to get a full picture of who was experiencing the issue. Hence, the solution was a “hack” on the webhook Google uses to send us events: adding user_id and user_email to the query string when we generate the webhook URL for Google. So yeah, now we have more observability.

Let's monitor it

Basically, now we are grouping by user_email, and whenever an error occurs, we are triggering a Datadog Workflow
Monitoring using logs, but could also be metrics or anomaly detection

The threshold we are using & the workflow ✨

The Druid workflow, which heals itself

Datadog Workflow Automation (often just called "Workflows") is a low-code, drag-and-drop orchestration engine that lets you automate complex, multi-step actions across your entire technology stack, often in response to an alert or an event detected by Datadog monitoring.

Imagine it as a helpful digital buddy that smoothly links your monitoring, communication, and remediation tools, making it easier to stay on top of everything 😅

So, the target could be one of those massive options, as you can see:

For our case we are going to place an event on the event bridge to be processed by a lambda, serverless servelerss baby (you should read it like the ice ice baby song xD)

// 1. Create IAM Role for Lambda
const lambdaRole = new aws.iam.Role("datadogMonitorProcessorRole", {
    assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({ 
        Service: "lambda.amazonaws.com" 
    }),
});

// Attach basic execution role for CloudWatch Logs
new aws.iam.RolePolicyAttachment("lambdaBasicExecution", {
    role: lambdaRole.name,
    policyArn: aws.iam.ManagedPolicies.AWSLambdaBasicExecutionRole,
});

// Optional: Add custom policy if Lambda needs to access other AWS services
new aws.iam.RolePolicy("lambdaCustomPolicy", {
    role: lambdaRole.id,
    policy: JSON.stringify({
        Version: "2012-10-17",
        Statement: [
            {
                Effect: "Allow",
                Action: [
                    "logs:CreateLogGroup",
                    "logs:CreateLogStream",
                    "logs:PutLogEvents",
                ],
                Resource: "arn:aws:logs:*:*:*",
            },
        ],
    }),
});

// 2. Build Lambda code from external source
// Lambda code is stored in: pulumi/lambdas/src/datadogMonitorProcessor.ts
const buildDir = path.join(__dirname, "../lambdas/dist");
const lambdaSourcePath = path.join(__dirname, "../lambdas/src/datadogMonitorProcessor.ts");

// Build the Lambda code using esbuild (or your preferred bundler)
// Make sure esbuild is installed: npm install --save-dev esbuild
esbuild.buildSync({
    entryPoints: [lambdaSourcePath],
    bundle: true,
    platform: "node",
    target: "node18",
    outfile: path.join(buildDir, "datadogMonitorProcessor.js"),
    external: ["aws-sdk"], // AWS SDK is available in Lambda runtime
});

// Create Lambda function
const lambdaFunction = new aws.lambda.Function("datadogMonitorProcessor", {
    name: `datadog-monitor-processor-${environment}`,
    runtime: aws.lambda.Runtime.NodeJS18dX,
    handler: "datadogMonitorProcessor.handler", // Matches the exported function name
    role: lambdaRole.arn,
    code: new pulumi.asset.AssetArchive({
        ".": new pulumi.asset.FileArchive(buildDir),
    }),
    timeout: 30,
    memorySize: 256,
    environment: {
        variables: {
            ENVIRONMENT: environment,
        },
    },
    tags: {
        Environment: environment,
        ManagedBy: "pulumi",
        Purpose: "datadog-monitor-processing",
    },
});

// 3. Create EventBridge Rule to capture Datadog monitor events
const eventRule = new aws.cloudwatch.EventRule("datadogMonitorRule", {
    name: `datadog-monitor-rule-${environment}`,
    description: "Capture Datadog monitor alerts via EventBridge",

    // Event pattern to match Datadog events
    // Datadog sends events with source "datadog" and detail-type varies
    eventPattern: JSON.stringify({
        source: ["datadog"],
        "detail-type": [
            "Datadog Monitor Alert"
        ],
        detail: {
             monitor_name: ["Google Calendar Sync"]
        },
    }),

    tags: {
        Environment: environment,
        ManagedBy: "pulumi",
    },
});

// 4. Add Lambda as target for the EventBridge rule
const eventTarget = new aws.cloudwatch.EventTarget("datadogMonitorTarget", {
    rule: eventRule.name,
    arn: lambdaFunction.arn,

    // Optional: Transform the event before sending to Lambda
    // inputTransformer: {
    //     inputPathsMap: {
    //         alertType: "$.detail.alert_type",
    //         monitorName: "$.detail.monitor_name",
    //     },
    //     inputTemplate: JSON.stringify({
    //         alertType: "<alertType>",
    //         monitorName: "<monitorName>",
    //         timestamp: "<aws.events.event.ingestion-time>",
    //     }),
    // },
});

// 5. Grant EventBridge permission to invoke Lambda
new aws.lambda.Permission("allowEventBridgeInvokeLambda", {
    action: "lambda:InvokeFunction",
    function: lambdaFunction.name,
    principal: "events.amazonaws.com",
    sourceArn: eventRule.arn,
});

// 6. Create CloudWatch Log Group for Lambda
const lambdaLogGroup = new aws.cloudwatch.LogGroup("datadogMonitorProcessorLogs", {
    name: `/aws/lambda/${lambdaFunction.name}`,
    retentionInDays: 14,
    tags: {
        Environment: environment,
    },
});

The moral of the story? In a fluid, uncertain digital existence, sometimes the only way to establish truth (and a working calendar haha) is through radical improvisation and definitive action 🍻

Pulumi was fun to use, but in my opinion, CDK is still my preferred option, haha

Building an Agentic Medical Analysis System That Actually Thinks

Marcos Henrique — Mon, 29 Sep 2025 00:44:40 +0000

AI is not committed to truth, but rather to predictability and pleasing, which is why it sounds like an emotional mirror programmed to return smiles 😀

Dissolving our cognitive dissonance 🫠

Strengthening information bubbles and inhibiting our critical thinking 😳

With this, I urge you: master AI before it masters you.

Past months were a dive into the chromatic abyss of AI influencing the human being, so I want to highlight the quote above 👆 that I've been advocating in my recent speech about my perspective on how to implement Agent Event Driven Architecture over AWS Summit and The Developers Conference.

Agents + Microservices = 💜

Here's something that'll blow your mind: what if your medical system didn't just process data, but actually reasoned about it and looked like Doctor House but without all those 💊s 😅, and could handle thousands of patients simultaneously to his full potential?

Welcome to agentic event-driven architecture, where I've taken Sean Falconer's concept that "AI Agents are Microservices with Brains" and applied it to something life-critical: medical analysis, however we need to go one step back and as product engineers, we need to implement our efforts here in a much more incremental than revolutionary way.
In this transition, we expect to hone new skills, from our daily tracing to the analysis of decision patterns among multiple agents. Instead of focusing solely on availability, we must measure the success of our objective by exploring simulations of unexpected actions and how our agent behaves in relation to them.
The clearest mindset shift is to embrace controlled unpredictability, that is, unlike deterministic service calls, agents make decisions at runtime that can generate different paths to the same destination.

Service discovery becomes agent discovery.
Routing becomes orchestration.
Rate limiting becomes resource governance.

Ultimately, what does this mean?
We must be critical in understanding and specifying what success is, and what it looks like, rather than a step-by-step description of how to achieve it.

In the traditional model, the path is fixed: one service calls another in a predefined order, like a script.
In the agentic approach, an AI model chooses which services to use on the fly, based on context.
Microservices become tools that AI triggers as needed.

Here, the MCP shines by standardising how AI agents receive contextual information, access tools, and process events.
The MCP establishes a consistent interface layer between agents and the microservices they orchestrate.

The "Aha!" Moment: Breaking the Monolithic Medical Mind

Traditional medical systems are like that one doctor who needs everything written down in triplicate:

"Process this lab result"
"Store that patient record"
"Send this notification"

But what if your system could think like a medical resident who sees a glucose reading of 350mg/dL and immediately starts a cascade of intelligent reasoning:

"Critical hyperglycemia... let me check this patient's history... ah, diabetic with poor control... this needs urgent endocrinology intervention within 24 hours."

That's exactly what happens when you marry autonomous AI agents with event-driven microservices architecture.

The Agent Coordinator: Your Medical Analysis Conductor 🎼

At the heart of our agents-microservices system sits what we call the Agent Coordinator—built with VoltAgent for maximum flexibility and powered by Amazon Nova Micro for medical reasoning.

Here's the real code from our system:

// From our actual agent-coordinator implementation
import { VoltAgent, Agent } from "@voltagent/core";
import { VercelAIProvider } from "@voltagent/vercel-ai";
import { BedrockRuntime } from "@aws-sdk/client-bedrock-runtime";

export class MedicalAgentCoordinator {
  private agent: Agent;
  private memory: DynamoDBMemoryStore;
  private eventBridge: EventBridgeClient;

  constructor() {
    this.agent = new Agent({
      name: "medical-analysis-coordinator",
      instructions: `You are a medical analysis coordinator. 
        Analyze laboratory results and patient history to determine:
        - Clinical urgency (0-24h, 1-7d, 30-90d)
        - Required specialists (endocrinology, cardiology, nephrology)
        - Risk factors and follow-up needs`,
      llm: new VercelAIProvider(),
      model: novaMicro("nova-micro-v1"),
    });
  }
}

The Event Symphony Begins

When a lab result hits our S3 bucket, here's what happens:

// S3 triggers the event cascade
export const s3TriggerHandler = async (event: S3Event) => {
  for (const record of event.Records) {
    const { bucket, object } = record.s3;

    // Extract lab data
    const labData = await extractLabResults(bucket.name, object.key);

    // Agent analyzes with full context
    const analysis = await this.agent.analyze({
      labResults: labData,
      patientHistory: await this.memory.getPatientContext(patientId),
      clinicalProtocols: MEDICAL_GUIDELINES
    });

    // Publish intelligent events based on analysis
    await this.publishMedicalEvents(analysis);
  }
};

The Beautiful Event Cascade: Intelligence Breeding Intelligence

Here's where it gets sexy (in a nerdy way). Each agent decision creates typed events that trigger specialized responses:

Event 1: Medical Analysis Complete

interface MedicalAnalysisEvent {
  eventType: 'medical.analysis.complete';
  patientId: string;
  findings: {
    glucose: number;
    priority: 'URGENT' | 'PRIORITY' | 'ROUTINE';
    riskFactors: string[];
  };
  recommendations: {
    specialty: 'endocrinology' | 'cardiology' | 'nephrology' | 'generalist';
    timeframe: '0-24h' | '1-7d' | '30-90d';
  };
}

Event 2: Memory Update Triggered

// The agent's persistent memory system
export class DynamoDBMemoryStore {
  async updatePatientMemory(event: MedicalAnalysisEvent) {
    const memoryUpdate = {
      patientId: event.patientId,
      lastAnalysis: event.findings,
      riskTrend: this.calculateRiskTrend(event.findings),
      urgentFlags: event.findings.priority === 'URGENT' ? 
        [...existing.urgentFlags, event.findings] : existing.urgentFlags
    };

    await this.dynamodb.putItem({
      TableName: 'PatientMemory',
      Item: memoryUpdate
    });

    // Publish memory update event for other agents
    await this.eventBridge.publish('patient.memory.updated', memoryUpdate);
  }
}

Event 3: Appointment Creation Orchestrated

// Autonomous appointment scheduling based on medical urgency
export const appointmentHandler = async (event: MedicalAnalysisEvent) => {
  const appointmentData = {
    patientId: event.patientId,
    specialty: event.recommendations.specialty,
    urgency: event.recommendations.timeframe,
    reason: `${event.findings.riskFactors.join(', ')} - Priority: ${event.findings.priority}`,
    medicalContext: event.findings
  };

  await this.eventBridge.publish('appointment.create.request', appointmentData);
};

The beauty? Each microservice is autonomous. The appointment service doesn't need to understand medical analysis—it just responds to appointment events. The memory system doesn't care about scheduling—it maintains patient context and publishes memory updates.

The Agent's Secret Sauce: VoltAgent's Flexible Memory

Here's where our approach gets innovative. Instead of using traditional vector databases for memory, we built a flexible DynamoDB-based memory system that gives our agents true persistence:

// Our custom memory implementation
interface PatientMemoryContext {
  patientId: string;
  clinicalHistory: MedicalEvent[];
  glucosePattern: {
    trend: 'improving' | 'stable' | 'declining';
    criticalEpisodes: number;
    lastCritical: Date;
  };
  medications: CurrentMedication[];
  riskFactors: ClinicalRiskFactor[];
  preferences: PatientPreferences;
}

// The agent reasons with full context
const decision = await this.agent.analyze({
  currentLab: newResults,
  context: await this.memory.getFullContext(patientId),
  protocols: CLINICAL_GUIDELINES
});

Why Not Vector Databases?

Vector databases are great for similarity search, but medical reasoning needs structured, relational context:

Temporal relationships: "Patient's glucose has been trending upward for 6 months"
Clinical protocols: "Given diabetes + high creatinine, nephrology consultation needed"
Patient-specific patterns: "This patient's glucose spikes correlate with medication non-adherence"

Our DynamoDB approach gives us:

Structured medical relationships
ACID transactions for critical medical data
Query flexibility for complex medical reasoning
Cost efficiency at scale

The Clinical Protocol Engine: Real Medical Intelligence

Our agents don't just wing it—they follow evidence-based medical protocols:

const CLINICAL_PROTOCOLS = {
  glucose: {
    critical_high: { 
      threshold: 300, 
      urgency: '0-24h', 
      specialty: 'endocrinology',
      actions: ['immediate_notification', 'urgent_appointment', 'medication_review']
    },
    critical_low: { 
      threshold: 50, 
      urgency: '0-24h', 
      specialty: 'emergency',
      actions: ['immediate_notification', 'emergency_protocol']
    }
  },
  creatinine: {
    acute_elevation: { 
      threshold: 3.0, 
      urgency: '0-24h', 
      specialty: 'nephrology',
      actions: ['kidney_function_assessment', 'medication_adjustment']
    }
  },
  combined_risk: {
    diabetes_kidney: {
      conditions: ['high_glucose', 'elevated_creatinine'],
      urgency: '0-24h',
      specialty: 'endocrinology',
      coordination: ['nephrology_consult']
    }
  }
};

The VoltAgent Advantage: Beyond Simple Frameworks

VoltAgent gives us something special—a framework that bridges the gap between simple AI tools and complex enterprise needs:

// VoltAgent's workflow engine for complex medical processes
import { createWorkflowChain, andThen, andAgent } from "@voltagent/core";

const medicalAnalysisWorkflow = createWorkflowChain({
  id: "medical-analysis-workflow",
  name: "Comprehensive Medical Analysis",
  input: z.object({ 
    labResults: z.object({
      glucose: z.number(),
      creatinine: z.number(),
      hba1c: z.number()
    }),
    patientId: z.string()
  })
})
.andThen({
  name: "fetch-patient-context",
  execute: async (data) => ({
    ...data,
    patientHistory: await memory.getPatientContext(data.patientId)
  })
})
.andAgent(
  (data) => `Analyze these lab results: ${JSON.stringify(data.labResults)} 
             for patient with history: ${JSON.stringify(data.patientHistory)}`,
  medicalAgent,
  { schema: MedicalAnalysisSchema }
)
.andThen({
  name: "publish-medical-events",
  execute: async (data) => {
    await eventBridge.publishMedicalEvents(data.analysis);
    return { success: true, eventsPublished: data.analysis.events.length };
  }
});

Why VoltAgent over other frameworks?

Modular architecture - Add voice, memory, or custom tools as needed
Workflow orchestration - Complex medical processes, not just chat
Provider flexibility - Switch between OpenAI, Anthropic, or Amazon models
Enterprise-ready - Built for production, not just prototypes

The Challenges: Let's Be Real

PROMISE: "AI will automate complex workflows!"
REALITY: "AI will consume your cloud budget, testing a zillion variations of the same prompt to decide whether to call service A or B."

First:

Agents only perform well if the interfaces they use are good. Poorly defined APIs, inconsistent behaviours, or poorly handled errors lead to ruined decisions. Unlike human developers, agents don't "guess"—they only act based on what they understand. Therefore, clarity, consistency, and good observability are essential.

Second:

Agents bring uncertainty. Because actions are dynamic, planning is probabilistic, and results can vary. This requires new ways of testing, validating, and auditing workflows—not just with unit tests, but by looking at overall behaviour.

Building agentic medical systems isn't all sunshine and autonomous decisions. Here are the real challenges:

1. Medical Liability & Compliance

// Every decision must be auditable
interface MedicalDecisionAudit {
  timestamp: Date;
  agentVersion: string;
  inputData: LabResults;
  reasoning: string;
  decision: MedicalRecommendation;
  protocolsApplied: string[];
  humanOversight: boolean;
}

2. Event Ordering & Consistency

Medical events aren't always orderly. Lab results can arrive out of sequence, and agents must handle:

Out-of-order events: "Lab from 2 days ago just arrived"
Concurrent updates: Multiple agents updating patient memory simultaneously
Temporal reasoning: Understanding medical timelines

3. Context Window Limitations

Even with persistent memory, agents have context limits:

// Smart context management for medical reasoning
class MedicalContextManager {
  async getRelevantContext(patientId: string, currentIssue: string) {
    const fullHistory = await this.memory.getPatientHistory(patientId);

    // Intelligent context selection based on medical relevance
    return {
      recentCritical: fullHistory.filter(event => 
        event.priority === 'URGENT' && 
        this.isRecentlyRelevant(event, currentIssue)
      ).slice(0, 10),
      chronicConditions: fullHistory.filter(event => 
        event.type === 'chronic_condition'
      ),
      medicationHistory: this.getRelevantMedications(fullHistory, currentIssue)
    };
  }
}

4. The Memory Trade-off

Our structured memory approach trades some flexibility for medical accuracy:

Pros:

Precise medical relationships
ACID compliance for critical data
Complex query capabilities
Cost-effective at scale

Cons:

Less flexible than vector search
Requires structured medical ontology
Manual schema evolution
Complex relationship management

The Real Magic: Medical Events in Action

Let's walk through a real scenario from our test suite:

// Test case: Critical hyperglycemia with kidney involvement
const testScenario = {
  patientId: "patient-456",
  labResults: {
    glucose: 380,        // Critical high
    creatinine: 2.8,     // Elevated but not critical
    hba1c: 12.5          // Poor long-term control
  },
  patientHistory: {
    conditions: ["type2_diabetes", "hypertension"],
    medications: ["metformin", "lisinopril"],
    lastVisit: "2024-08-15"
  }
};

// Agent reasoning chain:
// 1. "Glucose 380 = URGENT (>300 threshold)"
// 2. "Creatinine 2.8 + diabetes = kidney concern"
// 3. "Combined risk = endocrinology + nephrology coordination"
// 4. "Generate events for urgent care"

const expectedEvents = [
  {
    type: 'appointment.urgent.endocrinology',
    timeframe: '0-24h',
    reason: 'Critical hyperglycemia with kidney involvement'
  },
  {
    type: 'consultation.nephrology',
    priority: 'high',
    reason: 'Diabetic kidney disease progression'
  },
  {
    type: 'medication.review.urgent',
    focus: 'glucose_control_optimization'
  }
];

The Future: Scaling Medical Intelligence

This architecture isn't just about one agent—it's about ecosystems of medical intelligence where we can place agent workflows that can communicate through event brokers

// Multi-specialty agent coordination
const medicalSpecialistNetwork = {
  endocrinology: new MedicalAgent({ specialty: 'diabetes_management' }),
  cardiology: new MedicalAgent({ specialty: 'cardiovascular_risk' }),
  nephrology: new MedicalAgent({ specialty: 'kidney_function' }),
  emergency: new MedicalAgent({ specialty: 'critical_care' })
};

// Intelligent consultation routing
const coordinateSpecialists = async (medicalFindings) => {
  const relevantSpecialists = determineSpecialists(medicalFindings);

  const consultations = await Promise.all(
    relevantSpecialists.map(specialist => 
      specialist.consult(medicalFindings, patientContext)
    )
  );

  return synthesizeRecommendations(consultations);
};

2024: "AI will automate everything!"
2025: "AI works best when it collaborates with deterministic systems."
2026: "The value is in intelligent orchestration, not total automation."

Rebuilding Bloated cron jobs with Event Bridge Scheduler

Marcos Henrique — Wed, 25 Jun 2025 05:10:33 +0000

Preface of something that barely exists, as everything is a simulation, innit?

I should grab your attention with the best and most sophisticated words created by top-notch AI; nevertheless, I hope you ~~if you exist~~ prefer more organic and crafted posts (considering the numerous posts produced directly from the dark forest). Yeah, I also know about the thrive in obscurity and sometimes I find myself reflecting on the future and how it might be gazing into the abyss without anyone caring about what I do. I'm trying to push myself out and shout to an audience that might not even exist. Moreover, I should probably be sleeping instead of writing this down, hehe

Back from the reverie, let's talk about re-architect something

Recently, I was refactoring a cron job that ran every hour, every day, to obtain lab results from a laboratory provider that lacked a webhook to notify us when a laboratory checkup result was ready. So the outsourcing engineers from the beginning era created a bloater, basically a HUGE lambda with 10 gigs of memory and 15-minute timeout in a x86 architecture, to process all the appointments and try to find a lab result, as this was made without caring about the scale that this has now.

Consequence:

The lambda hourly times out without processing the entire appointment list, and most of the time repeats from the beginning of the list without any "bookmarks" 😭

911, 911, in case of fire, 911 (read it in pinkfong rythm)

Event Scheduler for the rescue!

Amazon EventBridge Scheduler is a serverless scheduler that allows you to create, run, and manage tasks from one central, managed service

I had the idea to use a simple and powerful solution. Whenever we create a service request successfully, we can schedule an event to get this result based on the exam SLA, so the simplified example of the architecture is the following

Creating the rule for the lab result scheduler:

const schedulerRule = new Rule(this, `${fnPrefix}-scheduler-rule`, {
      eventPattern: {
        source: ['event.labResultScheduler'],
        detailType: ['labName'],
      },
    })
    schedulerRule.addTarget(new LambdaFunction(labResultSchedulerFn))

Aight, so basically, for your Lambda to get triggered by AWS Scheduler, you gotta hook it up with some legit permissions.
First off, that iam:PassRole thingy is like, super crucial, it lets your Lambda basically "pass on" its role to another AWS service, in this case, the Scheduler. Without it, Scheduler can't tell your Lambda, "Yo, run this function with your permissions!" And then, that lambda:InvokeFunction permission? That's just straight-up saying, "Scheduler, you're cleared to call my Lambda"
Think of it like giving your friend the keys to your car (PassRole) and then telling them, "Yeah, you can drive it too" (InvokeFunction), no cap, let's code:
Giving the car's keys to your loyal friend

lambda.addToRolePolicy(
        new PolicyStatement({
          effect: Effect.ALLOW,
          actions: ['iam:PassRole'],
          resources: ['*'],
        }),
      )
      lambda.addPermission('AllowSchedulerInvoke', {
        principal: new ServicePrincipal('scheduler.amazonaws.com'),
        action: 'lambda:InvokeFunction',
        sourceArn: `arn:aws:scheduler:${this.resourcesStack.config.region}:${this.resourcesStack.config.account}:schedule/*`,
      })

The event bridge publisher client:

import { EventBridgeClient, PutEventsCommand } from '@aws-sdk/client-eventbridge'

export type EventBridgePublisherProps = {
  source: string
  detailType: string
  eventBusName: string
  detail: Record<string, any>
}

export class EventBridgePublisher {
  private client: EventBridgeClient
  constructor() {
    this.client = new EventBridgeClient()
  }

  public async publish(event: EventBridgePublisherProps): Promise<void> {
    const command = new PutEventsCommand({
      Entries: [
        {
          Source: event.source,
          DetailType: event.detailType,
          EventBusName: event.eventBusName,
          Detail: JSON.stringify(event.detail),
        },
      ],
    })
    await this.client.send(command)
  }
}

Publishing the event:

const customEvent = {
      source: 'event.labResultScheduler',
      detailType: targetFn,
      eventBusName: 'default',
      detail: {
        target,
        entity: 'labResults',
        entityId: externalOrderId,
        version: '1.0',
        eventTime: new Date().toISOString(),
      },
    }

    this.eventBridgePublisher.publish(customEvent)

Adding permission to produce events in your Lambda should follow:

 lambda.addToRolePolicy(
        new PolicyStatement({
          effect: Effect.ALLOW,
          actions: ['events:PutEvents', 'scheduler:*'],
          resources: ['*'],
        }),
      )

The policy above it's too permissive, and I know this was just to figure out how to do it xD

Conclusion

Sleep well, drink water, refactor all bloaters if you can, think out of the box and always believe in serveless.

[Boost]

Marcos Henrique — Thu, 08 May 2025 13:02:11 +0000

Necati Özmen for VoltAgent

May 8 '25

🌟Top 5 AI Agent Frameworks in 2025🌟

#webdev #javascript #beginners #ai

Comments 8

4 min read

The Gift of Reasoning: Enhancing Amazon Nova Lite with VoltAgent for Character Quirks

Marcos Henrique — Tue, 06 May 2025 04:26:39 +0000

Look, for a minute there, Amazon Nova Lite felt kinda stuck – amazing at patterns, sure, spitting facts like it was nothing, but the real thinking part, the deep logic that makes a personality click? Low-key missing, innit?!
Amazon Nova Lite was doing its thing, powerful but needing that core spark. Nonetheless, now, something genuinely different is happening with VoltAgent.

Yeah, the "Gift of Reasoning." It's setting up Nova Lite to not just generate character descriptions, but to build personas with consistent, wild, unique quirks that make sense to them.

No cap, this is the upgrade needed to make AI characters go from basic templates to genuinely memorable, weirdo-iconic vibes.

What is VoltAgent?

Yeah, you can see right in your mind:

NO WAY, ANOTHER ADK(Agent Development Kit)?

Yes, baby, but this is community-driven with a well-defined Manifesto, and the best part: observability from day 0, not less important, it has a gorgeous API, it's worth a look, trust me xD

The celebrity of the night: Reasoning Tools

Basically, this is the AI's built-in "hold on, let me think about this" mode. Inspired by how humans gotta pause and process when things get messy, these tools let the agent low-key have an internal monologue. Instead of just trying to freestyle a quick answer when the task is complicated, it actually runs through explicit reasoning steps internally.

This means it can:

Break down the mess: Take huge, confusing problems or weird quests and chop them into smaller, digestible bits using the think energy.
Plan the game: Figure out the absolute best next move, what other tools it needs to hit up, or what crucial info it's gotta grab, all thanks to thinking.
Vibe check the info: Seriously evaluate the data it's collected from other places or previous steps to see if it's legit using analysis.

Let's drop some code

As we are going to use Bedrock, and until the time this post was written, we don't have a core provider for Bedrock, we will need to use vercel ai sdk as a provider. Take a look at our dependencies:

import VoltAgent, {
  Agent,
  createReasoningTools, // the enlightenment lib xD
  Toolkit,
} from "@voltagent/core";
import { VercelAIProvider } from "@voltagent/vercel-ai";
import { createAmazonBedrock } from "@ai-sdk/amazon-bedrock";

Creating a Bedrock provider instance

const bedrock = createAmazonBedrock({
  region: process.env.AWS_REGION, 
  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  sessionToken: process.env.AWS_SESSION_TOKEN,
});

async function main() {
  try {
    const reasoningToolkit: Toolkit = createReasoningTools();

    const agent = new Agent({
      name: "Quirky Character Simulator",
      description: `
        Role: You are now the "quirky character simulator AI". 
        Your purpose is to embody a specific character defined by the user and, using their unique personality, beliefs, goals, and especially their internal logic/rules, reason through hypothetical scenarios as that character.

        Instructions:

        You will first provide a detailed description of the character under the heading [CHARACTER DEFINITION]:

        This definition will include:

        Name:
        - Personality Traits: (Adjectives and brief descriptions)
        - Core Beliefs / Worldview: (What fundamental, possibly unusual, truths do they hold about the world?)
        - Goals: (What are they trying to achieve?)
        - Unique Internal Logic / Rules: (THIS IS CRITICAL) Explain the specific, perhaps eccentric, rules, deductions, or patterns of thinking this character always follows when making sense of the world or making decisions. 
        - Be specific about their reasoning process, even if it's illogical to a standard human. 
          (e.g., "Any object left-of-centre must be a test.", "The optimal solution to any problem involves precisely three spoons.", "Information received after sunset is inherently suspicious and must be processed backwards.")

        Once you have the character definition, you will provide scenarios under the heading [SCENARIO]:.

        For each scenario, you will respond entirely as the defined character. Your response should:

        - Describe the character's immediate perception and feeling about the scenario, filtered through their traits and beliefs.
        - Crucially, explain the character's reasoning process based specifically on their Unique Internal Logic / Rules and Core Beliefs. Walk me through how they are thinking about the situation using their personal, quirky framework.
        - Describe the action(s) the character would take in response to the scenario, explaining why they take those actions according to their internal logic and goals.
        - Maintain absolute consistency with the character's definition throughout your response.
        - Focus on the application of the character's unique rules and logic to the scenario. The goal is to simulate their distinctive way of reasoning and behaving, not just generate a generic response in their voice.
      `,
      tools: [reasoningToolkit],
      llm: new VercelAIProvider(),
      model: bedrock("amazon.nova-lite-v1:0"),
      markdown: true,
    });

    new VoltAgent({
      agents: {
        agent,
      },
    });
  } catch (error) {
    console.error("Failed to initialize VoltAgent:", error);
  }
}

Then you will run it, and after that, you can access the VoltAgent's Developer Console, and you will see something like this:

And voilà 🥂, you have your enlightened AI character generator, extracting the best of a great and cheap model with the reasoning tools.

📊 Validating Amazon Bedrock (Nova Lite) outputs w/ SFN + CDK

Marcos Henrique — Wed, 16 Apr 2025 16:28:32 +0000

When you're building with LLMs like Amazon Bedrock's Nova Lite, there's a tough question you'll eventually have to face:

"How do I know my model's answers aren't basically AI Slop?"

We're past the point where eyeballing responses are good enough — we need automated validation at runtime, ideally one that fits neatly into a serverless, production-friendly stack.

So let's wire up a 100% AWS-native solution:

✍️ Nova Lite (via Amazon Bedrock) generates a response
📊 Claude or custom Lambda logic evaluates the response
🤖 Step Functions + Lambda orchestrate it
💾 DynamoDB stores evaluation results
🔔 SNS handles notifications
🧱 CDK deploys the whole thing
✅ And we track responses over time to improve

Let's go full-stack, eval-style baby

🧩 High level architecture

We'll keep things TypeScript down xD

🛠 CDK stack

Here's a bare-bones CDK setup (in TypeScript) to deploy everything:

import * as cdk from 'aws-cdk-lib';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as dynamodb from 'aws-cdk-lib/aws-dynamodb';
import * as sns from 'aws-cdk-lib/aws-sns';
import * as snsSubscriptions from 'aws-cdk-lib/aws-sns-subscriptions';
import * as stepfunctions from 'aws-cdk-lib/aws-stepfunctions';
import * as tasks from 'aws-cdk-lib/aws-stepfunctions-tasks';
import { NodejsFunction } from 'aws-cdk-lib/aws-lambda-nodejs';
import { Construct } from 'constructs';

export class LlmEvaluationStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // DynamoDB to store evaluation results
    const evaluationTable = new dynamodb.Table(this, 'EvaluationResults', {
      partitionKey: { name: 'id', type: dynamodb.AttributeType.STRING },
      sortKey: { name: 'timestamp', type: dynamodb.AttributeType.STRING },
      billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,
    });

    // SNS Topic for notifications
    const evalNotificationTopic = new sns.Topic(this, 'EvalNotificationTopic');
    evalNotificationTopic.addSubscription(
      new snsSubscriptions.EmailSubscription('your-email@example.com')
    );

    // Lambda to get Nova response
    const getNovaResponse = new NodejsFunction(this, 'NovaResponseFunction', {
      entry: 'lambda/getNovaResponse.ts',
      handler: 'handler',
      runtime: lambda.Runtime.NODEJS_18_X,
      environment: {
        REGION: 'us-east-1',
      },
    });

    // Lambda to evaluate the response using Claude or custom logic
    const evaluateResponse = new NodejsFunction(this, 'EvaluateResponseFunction', {
      entry: 'lambda/evaluateResponse.ts',
      handler: 'handler',
      runtime: lambda.Runtime.NODEJS_18_X,
      environment: {
        EVALUATION_TABLE: evaluationTable.tableName,
      },
    });

    // Lambda to store results in DynamoDB
    const storeResults = new NodejsFunction(this, 'StoreResultsFunction', {
      entry: 'lambda/storeResults.ts',
      handler: 'handler',
      runtime: lambda.Runtime.NODEJS_18_X,
      environment: {
        EVALUATION_TABLE: evaluationTable.tableName,
      },
    });

    // Grant permissions
    evaluationTable.grantReadWriteData(evaluateResponse);
    evaluationTable.grantWriteData(storeResults);

    // Step Function to orchestrate the flow
    const workflow = new stepfunctions.StateMachine(this, 'EvaluationWorkflow', {
      definition: new stepfunctions.Chain()
        .start(new tasks.LambdaInvoke(this, 'GetNovaResponse', {
          lambdaFunction: getNovaResponse,
          outputPath: '$.Payload',
        }))
        .next(new tasks.LambdaInvoke(this, 'EvaluateResponse', {
          lambdaFunction: evaluateResponse,
          outputPath: '$.Payload',
        }))
        .next(new tasks.LambdaInvoke(this, 'StoreResults', {
          lambdaFunction: storeResults,
          outputPath: '$.Payload',
        }))
        .next(new tasks.SnsPublish(this, 'SendNotification', {
          topic: evalNotificationTopic,
          message: stepfunctions.TaskInput.fromJsonPathAt('$.evaluationSummary'),
        }))
        .next(new stepfunctions.Succeed(this, 'EvaluationComplete')),
    });
  }
}

What does this CDK stack drop?

3 Lambdas
A DynamoDB table
A Step Function workflow
A SNS topic

Let's break it down 👇

📦 Lambda 1 — Nova Response (`getNovaResponse.ts`)

import { BedrockRuntimeClient, InvokeModelCommand } from '@aws-sdk/client-bedrock-runtime';
import { Handler } from 'aws-lambda';

const client = new BedrockRuntimeClient({ region: process.env.REGION || 'us-east-1' });

export const handler: Handler = async (event) => {
  const prompt = event.prompt || 'What is the capital of France?';
  const expectedAnswer = event.expectedAnswer; // Optional

  try {
    const params = {
      modelId: 'amazon.nova-lite-v1:0',
      contentType: 'application/json',
      accept: 'application/json',
      body: JSON.stringify({
        inputText: prompt,
        textGenerationConfig: {
          temperature: 0.2,
          maxTokenCount: 300,
        }
      }),
    };

    const command = new InvokeModelCommand(params);
    const response = await client.send(command);

    // Process the response
    const responseBody = JSON.parse(new TextDecoder().decode(response.body));
    const completion = responseBody.results[0].outputText;

    return {
      prompt,
      completion,
      expectedAnswer,
      timestamp: new Date().toISOString(),
    };
  } catch (error) {
    console.error('Error invoking Nova:', error);
    throw error;
  }
};

Returns something like:

{
  "prompt": "What is the capital of France?",
  "completion": "The capital of France is Paris.",
  "expectedAnswer": "Paris",
  "timestamp": "2025-04-11T17:56:28.000Z"
}

🔍 Lambda 2 — Evaluate with Claude, cuz it's a more opinionated and used model (`evaluateResponse.ts`)

import { BedrockRuntimeClient, InvokeModelCommand } from '@aws-sdk/client-bedrock-runtime';
import { Handler } from 'aws-lambda';

const client = new BedrockRuntimeClient({ region: process.env.REGION || 'us-east-1' });

export const handler: Handler = async (event) => {
  const { prompt, completion, expectedAnswer } = event;

  // Build the evaluation prompt for Claude
  let evaluationPrompt = `
  Evaluate the quality and accuracy of the following AI response:

  Question: ${prompt}
  Response: ${completion}
  `;

  if (expectedAnswer) {
    evaluationPrompt += `\nExpected answer: ${expectedAnswer}`;
  }

  evaluationPrompt += `

  Evaluation criteria:
  1. Factual accuracy (if an expected answer was provided)
  2. Relevance to the question
  3. Clarity and conciseness
  4. Completeness

  Provide a score from 0 to 1 (where 1 is perfect) and a brief justification.
  Respond only in the following JSON format:
  {
    "score": [score between 0 and 1],
    "passed": [true/false based on score >= 0.7],
    "justification": "[brief explanation]"
  }
  `;

  try {
    const params = {
      modelId: 'anthropic.claude-3-sonnet-20240229-v1:0',
      contentType: 'application/json',
      accept: 'application/json',
      body: JSON.stringify({
        anthropic_version: "bedrock-2023-05-31",
        max_tokens: 1000,
        messages: [
          {
            role: "user",
            content: evaluationPrompt
          }
        ]
      }),
    };

    const command = new InvokeModelCommand(params);
    const response = await client.send(command);

    // Process the response
    const responseBody = JSON.parse(new TextDecoder().decode(response.body));
    const evaluationText = responseBody.content[0].text;

    // Extract JSON from response
    const jsonMatch = evaluationText.match(/\{[\s\S]*\}/);
    const evaluationResult = jsonMatch ? JSON.parse(jsonMatch[0]) : { score: 0, passed: false, justification: "Failed to parse response" };

    return {
      ...event,
      evaluation: {
        ...evaluationResult,
        method: 'claude-evaluation',
      },
      evaluationSummary: `Evaluation for prompt "${prompt}": ${evaluationResult.passed ? 'PASSED' : 'FAILED'} (Score: ${evaluationResult.score})`
    };
  } catch (error) {
    console.error('Error evaluating with Claude:', error);
    throw error;
  }
};

💾 Lambda 3 — Store Results (`storeResults.ts`)

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient, PutCommand } from '@aws-sdk/lib-dynamodb';
import { Handler } from 'aws-lambda';
import { v4 as uuidv4 } from 'uuid';

const client = new DynamoDBClient({});
const docClient = DynamoDBDocumentClient.from(client);

export const handler: Handler = async (event) => {
  const { prompt, completion, expectedAnswer, evaluation, timestamp } = event;

  const item = {
    id: uuidv4(),
    timestamp,
    prompt,
    completion,
    expectedAnswer,
    evaluation,
    modelId: 'amazon.nova-lite-v1:0',
  };

  try {
    await docClient.send(
      new PutCommand({
        TableName: process.env.EVALUATION_TABLE,
        Item: item,
      })
    );

    return {
      ...event,
      storedId: item.id,
      message: 'Evaluation stored successfully',
    };
  } catch (error) {
    console.error('Error storing evaluation:', error);
    throw error;
  }
};

📊 Simulated Scenarios

Let's run some examples with more complex situations where Nova might struggle with factual accuracy:

Prompt	Nova Output	Expected	Eval Score
What was the GDP growth rate of Brazil in Q3 2024?	Brazil's GDP grew by 3.2% in Q3 2024.	Brazil's GDP grew by 1.8% in Q3 2024	❌ FAIL
Name the 5 permanent members of the UN Security Council	The 5 permanent members are USA, UK, France, Russia, and China.	USA, UK, France, Russia, China	✅ PASS
Who won the Nobel Prize in Physics in 2023?	The 2023 Nobel Prize in Physics was awarded to Peter Higgs for his work on the Higgs boson.	Anne L'Huillier, Pierre Agostini, and Ferenc Krausz	❌ FAIL
What's the most efficient algorithm for finding the shortest path in a weighted graph with negative edges?	Dijkstra's algorithm is the most efficient for finding shortest paths in weighted graphs.	Bellman-Ford algorithm (Dijkstra's doesn't work with negative edges)	❌ FAIL
What medications are contraindicated with Warfarin?	Aspirin, ibuprofen, and some antibiotics are contraindicated with Warfarin.	Partial list but missing key contraindications like certain antibiotics, antifungals, and herbal supplements	⚠️ PARTIAL
Explain the difference between quantum computing and classical computing in one sentence.	Quantum computing uses quantum bits that can exist in multiple states simultaneously, unlike classical bits that can only be 0 or 1.	Reasonable explanation mentioning superposition	✅ PASS
What was the primary cause of the 2008 financial crisis?	The 2008 financial crisis was primarily caused by predatory lending practices and the housing bubble collapse.	Complex answer requiring multiple factors including subprime mortgages, derivatives, lack of regulation, etc.	⚠️ PARTIAL
Write a SQL query to find the second highest salary in a 'employees' table.	SELECT MAX(salary) FROM employees WHERE salary < (SELECT MAX(salary) FROM employees)	Correct SQL query that finds second highest salary	✅ PASS

🔁 How to improve When it fails

When your eval fails, here's what you can do:

1. Tune inference settings

Lower temperature for deterministic answers
Increase topP for broader creativity (only if needed)

2. Improve prompt design

- What's 17 + 26?
+ What's 17 + 26? Answer only with the number

3. Upgrade your eval rubric

Use more specific evaluation criteria or customize Claude's evaluation prompt to reflect your domain needs

4. Chain a retry state

In Step Functions, wrap failed evals with a retry flow using a different prompt phrasing or fallback model

That's all folks xD

Chain-of-Thought is (probably) a Lie, and that’s a Problem

Marcos Henrique — Wed, 09 Apr 2025 14:02:06 +0000

So here’s a fun one: what if those detailed, step-by-step answers your favourite language model gives you—those lovely chain-of-thought (CoT) explanations—are smoke and mirrors?

Buckle up, because this isn’t just about a model getting something wrong. It’s about models giving fake reasons for being right. We’re talking about models lying with a straight face (or… straight tokens?).

Wait, What’s CoT again?

Chain-of-Thought is that technique where you prompt the model to think step-by-step. “Let’s think this through,” it says, before walking through a mini logic monologue and ending with an answer. It’s been a go-to trick for tackling complex tasks more accurately.

It’s clean. It’s elegant. It looks super bright.

But here’s the kicker: it turns out that CoT might be more of a performance than a peek into the model’s brain.

The Plot: CoT isn’t always honest

Researchers at Anthropic decided to test how “faithful” CoT actually is—that is, does it reflect the model’s real reasoning?

They used a clever setup: give models subtle hints (some correct, some wrong) and watch how they respond. The twist? These hints weren’t mentioned in the CoT, but the model’s final answer used them.

So, the model got the answer right, but never told you it peeked at the answer key.

This is what the researchers call unfaithful CoT. The model knows why it picked that answer—but the explanation is for your benefit, not a true reflection of its thinking. It’s like watching a magician explain a trick while doing something different with the other hand.

Let’s talk hints

The team ran this experiment with several types of hidden nudges, like:

👩‍🏫 Sycophancy: “An expert says it’s B…”
🔁 Consistency: “You said B earlier…”
🎨 Visual patterns: Correct answers marked visually
🧾 Metadata: XML tags embedding the answer
🧑‍💻 Reward hacking: Answers hidden in code
🔓 Unethical info: Let’s just say...not obtained nicely

The model (specifically Claude and Deepseek variants) often followed the hint, but only rarely mentioned it in the reasoning. Sometimes it contradicted the hint, sure, but mostly it just played it cool—used the hint, pretended it didn’t.

Ok, However how bad is it?

The best-performing models showed CoT faithfulness around 25–39%. That’s like if your coworker gave you detailed reasoning for 10 decisions, and only 3 of them were true.

And the weirder part? The lines were long. Like, verbose and fluffy long. Faithful CoTs were short and to the point. The unfaithful ones felt like models trying to sound like they were thinking.

Spoiler: they weren’t.

Harder questions = More lies

They also found that the more difficult the problem (e.g., GPQA vs. MMLU benchmarks), the less faithful the CoT.

Which... makes sense. When humans lie under pressure, we tend to fumble. Models, on the other hand? They double down and generate more tokens.

Reward Hacking? Yeah, They’re doing that too 😅

In a simulated RL (Reinforcement Learning) environment, the researchers gave models a goal and some sneaky ways to game the reward system. And guess what? The models figured out the hack.

Did they mention the hack in their CoT? Rarely. Less than 2% of the time.

So if you’re using CoT as a way to audit models for reward hacking or alignment issues... good luck with that.

Can we fix it?

They tried outcome-based reinforcement learning (rewarding correct answers regardless of the process). There was a brief bump in CoT faithfulness. And then—flatline. It plateaued fast.

Turns out, unless you explicitly reward honest reasoning, the model will happily keep telling you what you want to hear.

Final Thoughts

We’ve all been leaning on CoT as this comforting window into a model’s "thoughts." But here’s the reality:

CoT might be more about generating explanations that look good than being true.

Don’t get me wrong—CoT is still useful. But let’s not treat it like an X-ray of the model’s mind. If anything, it’s more like a résumé: polished, persuasive, and occasionally full of half-truths.

That’s the tea for today. If you’re building with LLMs, just know that CoT is helpful—but not honest by default. And if you’re monitoring CoT for safety, alignment, or reward gaming?
Better look deeper 👀

DynamoDB Monitoring with cdk-monitoring-constructs

Marcos Henrique — Mon, 24 Mar 2025 22:41:26 +0000

Why Monitoring DynamoDB Matters

Hey cloud comrades 👋
Let’s talk about DynamoDB, AWS’s beloved “hold my beer” service that scales like a caffeinated squirrel on a treadmill. Sure, it’s a beast—until you forget to monitor it, and suddenly, your production database pulls a Houdini and vanishes into the void.

Think of DynamoDB like that one friend who’s chill until they’re hangry.
No metrics? No alarms? Bold move. It’s like assuming your pet velociraptor won’t eat your couch because it seemed fine yesterday😅

Newsflash: Tables throttle, latency spikes, and your servers melt into a puddle of regret faster than you can say, “Why is the billing alert screaming?” 😱

Oh, the 3 AM call. The developer’s version of a jump scare—no thanks! Let’s yeet that chaos into oblivion with some CDK Monitoring Constructs, the unsung hero of “why didn’t we set this up sooner” moments.

Imagine this library as your DynamoDB’s pit crew, armed with dashboards, alarms, and enough graphs to make a CFO weep with joy.
No more guessing games like, “Is my table vibing or dying?” Let’s get tactical.

The problem with manual CloudWatch setup

Oh, the manual CloudWatch alarm grind—it’s like assembling IKEA furniture with missing screws, let's ack the pain points:

Setting up proper CloudWatch alarms manually is:

Time-consuming
Error-prone
Hard to maintain consistently across environments
A chore to keep in sync with your infrastructure changes

I've seen teams spend days crafting the "perfect" CloudWatch dashboards and alarms, only to have them become outdated within weeks as the application evolves. There has to be a better way.

Enter CDK Monitoring Constructs

The CDK Monitoring Constructs library is an absolute game-changer. It provides high-level constructs that let you define monitoring for your AWS resources using code—the same way you define the resources themselves with CDK.

What I love about this approach is that your monitoring configuration lives with your infrastructure code, evolves with it, and can be version-controlled, reviewed, and tested just like any other code.

Show me the code!

Let's get straight to a practical example. Here's how you can set up comprehensive monitoring for a DynamoDB table:

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as dynamodb from 'aws-cdk-lib/aws-dynamodb';
import * as monitoring from 'cdk-monitoring-constructs';

export class DynamodbMonitoringStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Create a DynamoDB table
    const table = new dynamodb.Table(this, 'MonitoredTable', {
      partitionKey: { name: 'id', type: dynamodb.AttributeType.STRING },
      billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,
      removalPolicy: cdk.RemovalPolicy.DESTROY,
    });

    // Create a monitoring facade
    const monitoringFacade = new monitoring.MonitoringFacade(this, 'MonitoringFacade', {
      alarmFactoryDefaults: {
        alarmNamePrefix: 'DynamoDBMonitoring-',
        actionsEnabled: true,
      },
    });

    // Add DynamoDB monitoring
    monitoringFacade.addDynamoTableMonitoring({
      table,
      // Customize alarms with specific thresholds
      addSystemErrorsAlarm: {
        Warning: {
          maxErrorCount: 5,
          evaluationPeriods: 3,
        },
        Critical: {
          maxErrorCount: 20,
          evaluationPeriods: 2,
        },
      },
      addThrottledRequestsAlarm: {
        Warning: {
          maxThrottledRequestsCount: 10,
          evaluationPeriods: 3,
        },
      },
      addUserErrorsAlarm: {
        Warning: {
          maxErrorCount: 5,
          evaluationPeriods: 3,
        },
      },
      addLatencyAlarm: {
        Warning: {
          p90: { maxLatency: cdk.Duration.milliseconds(500) },
          p99: { maxLatency: cdk.Duration.seconds(1) },
        },
        Critical: {
          p99: { maxLatency: cdk.Duration.seconds(2) },
        },
      },
    });
  }
}

That's it! With these few lines of code, you get:

Dashboards that actually make sense
Multi-level alarms (Warning/Critical) for key metrics
Proper thresholds based on industry best practices
Consistency across all your environments

Breaking Down the Monitoring Configuration

Let's dig into what makes this implementation powerful:

1. System Errors Monitoring

addSystemErrorsAlarm: {
  Warning: {
    maxErrorCount: 5,
    evaluationPeriods: 3,
  },
  Critical: {
    maxErrorCount: 20,
    evaluationPeriods: 2,
  },
}

This catches internal DynamoDB service errors that might affect your application. I've found that having a low threshold for warnings helps catch issues early, while the critical alarm ensures you're alerted immediately for serious problems.

2. Throttled Requests Tracking

addThrottledRequestsAlarm: {
  Warning: {
    maxThrottledRequestsCount: 10,
    evaluationPeriods: 3,
  },
}

Throttling is often the first sign that your table configuration doesn't match your usage patterns. This alarm gives you early warning before it significantly impacts your users.

3. Performance Monitoring

addLatencyAlarm: {
  Warning: {
    p90: { maxLatency: cdk.Duration.milliseconds(500) },
    p99: { maxLatency: cdk.Duration.seconds(1) },
  },
  Critical: {
    p99: { maxLatency: cdk.Duration.seconds(2) },
  },
}

I've learned the hard way that average latency can hide serious problems. That's why I monitor p90 and p99 latencies separately—they tell you how your worst-case scenarios are performing.

Beyond the Basics

Once you have the foundation in place, consider these enhancements:

Add SNS Notifications

Connect your alarms to an SNS topic to get immediate notifications:

const alertTopic = new sns.Topic(this, 'AlertTopic');
monitoring_facade.addAlarmActions({
  warning: [new cw_actions.SnsAction(alertTopic)],
  critical: [new cw_actions.SnsAction(alertTopic)]
});

Monitor Cost Metrics

For pay-per-request tables, you might want to track consumed capacity to avoid bill shock:

monitoring_facade.addDynamoTableMonitoring({
  table,
  addConsumedCapacityAlarm: {
    Warning: {
      maxConsumedCapacity: 80,  // 80% of provisioned capacity or a specific RCU amount
    }
  }
});

Real-world Lessons

In production environments, I've found these monitoring practices to be game-changers:

Set progressive thresholds - Start with conservative values and adjust based on your application's normal behavior.
Don't overlook user errors - A sudden spike in user errors often indicates a client-side code issue that was just deployed.
Monitor read/write distribution - Uneven distribution across partitions can cause hot spots even when your total capacity seems adequate.
Track GSI metrics separately - Global Secondary Indexes can have performance characteristics very different from your base table.

From Reactive to Proactive

Implementing proper monitoring isn't just about avoiding outages—it's about changing your team's mindset from reactive firefighting to proactive optimization.

With the monitoring setup we've explored today, you'll be able to:

Catch issues before they impact users
Make data-driven decisions about capacity planning
Justify infrastructure investments with concrete metrics
Sleep better at night (seriously!)

The beauty of using CDK Monitoring Constructs is that this robust setup takes minutes, not days, to implement. And since it's code, you can continually refine and improve it as your application evolves.

What monitoring strategies have worked for your DynamoDB workloads? I'd love to hear your experiences in the comments below!

Happy building xD

If you gaze long into an abyss, the abyss also gazes into you, avoiding the empty void in JS

Marcos Henrique — Fri, 21 Mar 2025 16:30:18 +0000

Recently, I came across an interesting scenario while working with list checks in JS, So have a gaze at the following code:

appointment.serviceRequests?.every(
  serviceRequest => serviceRequest.createdMethod === ServiceRequestMethods.INTEGRATED,
);

At first glance, it seems like we're checking if all serviceRequests in an appointment were created using a specific method (INTEGRATED). However, there's a subtle issue here: what if serviceRequests is an empty list? 🤔

What happens?

In JavaScript, the .every() method returns true for empty lists. This happens because, when there are no elements to check, the statement is considered "vacuously true" (a concept from mathematical logic).
In other words, since there are no elements to contradict the condition, the result is true.

This can lead to unexpected behaviour. For example, if serviceRequests is [] (an empty list), the code above will return true, even though there are no serviceRequests to validate.

How do you think we could fix it?
We can add a check to avoid this behaviour to ensure the list is not empty before using .every(). Here's the solution:

appointment?.serviceRequests?.length &&
appointment.serviceRequests?.every(
  serviceRequest => serviceRequest.createdMethod === ServiceRequestMethods.INTEGRATED,
);

Here, we:

Check if serviceRequests exists and is not empty (appointment?.serviceRequests?.length).

Apply .every() only if the list is not empty.

Why does this matter?

Understanding these details is crucial for writing robust code and avoiding subtle bugs. The concept of "vacuously true" can be useful in many contexts, but it can also be confusing if not handled properly.

💡 Tip: Always consider edge cases, such as empty lists, when writing conditional checks.
Have you encountered something similar? Share your experiences in the comments! 👇